Grouping files and 'splitting' by number and copying them inside folders - batch-processing

I have 5000+ files inside a directory.
I want to manage these files 'splitting' every 500 files so first pack is copied inside folder1, pack2 of other 500 files is copied in folder2 and so on.
Which .bat script can I use ?

#ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
SET "sourcedir=U:\sourcedir\t w o"
SET "destdir=U:\destdir"
SET /a destcount=0
SET /a maxcount=5
SET /a filecount=maxcount
FOR /f "delims=" %%a IN (
'dir /b /a-d "%sourcedir%\*" '
) DO (
SET /a filecount +=1
IF !filecount! geq %maxcount% (
SET /a filecount=0
SET /a destcount +=1
MD "%destdir%\folder!destcount!"
)
ECHO(COPY "%sourcedir%\%%a" "%destdir%\folder!destcount!\"
)
GOTO :EOF
You would need to change the settings of sourcedir and destdir to suit your circumstances.
maxcount sets the number of files to assign to a group.
The required COPY commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO(COPY to COPY to actually copy the files. Append >nul to suppress report messages (eg. 1 file copied)
Essentially, count the number of transfers and increment the destination-directoryname each time a group is complete, using delayedexpansion to access the run-time values of the counters

Related

batch script for file system: use of for-loop variable in another nested for-loop

the last 4-5 hours I spent trying to get my script fixed, and tried several things, but it still doesn't work as supposed. Here is the working part:
#echo off
rem automatically sort MP3s into an existing structure or add folders if needed
rem example of MP3-file pattern: Wu-Tang Clan - Gravel Pit (LP Version Clean).mp3
SETLOCAL
rem set variables
SET "source=g:\music\folder"
SET "str=Wu-Tang Clan - Gravel Pit (LP Version Clean).mp3"
SET "test=%source%\%str%"
rem split the filename from the remaining path. Take the left part
rem and assign it to %band% while the right part gets assigned to %song%
FOR /f "delims=" %%i IN ("%test%") DO SET "result=%%~nxi"
SET "band=%result: - =" & SET "song=%"
rem If there is no such folder (%band%), create it
IF not exist "%source%\%band%" MD "%source%\%band%"
rem As soon as there definetely is a fitting folder
rem move the file over there.
MOVE /-Y "%source%\%result%" "%source%\%band%"
The next step would be to enhance the code by a for-loop so I don't have to edit the script for every single one of my 3k+ files ;)
I tried really hard to get this code going, but failed:
#echo off
SETLOCAL
SET "source=g:\music\folder"
FOR %%f IN (%source%\*.mp3) DO (
echo %%f
FOR /f "delims=" %%i IN ("%%f") DO SET "result=%%~nxi"
SET "band=!result: - =" & SET "song=%"
IF not exist "%source%\%band%" MD "%source%\%band%"
MOVE /-Y "%source%\%result%" "%source%\%band%"
)
pause
So I added a for-loop and %%f was correctly filled at first and wasn't in the next for-loop.
FOR %%f IN (%source%\*.mp3) DO (
echo %%f
That resulted in: "g:\music\folder\Wu-Tang Clan - Gravel Pit (LP Version Clean).mp3"
Like it was supposed to. But after that
FOR /f "delims=" %%i IN ("%%f") DO SET "result=%%~nxi"
result and every following variable was empty, always.
I tried to fix it with a second variable as a 'helper':
FOR %%f IN (%source%\*.mp3) DO (
SET "helper=%%f"
FOR /f "delims=" %%i IN ("%helper%") DO SET "result=%%~nxi"
Even added 'enabledelayedexpansion' after I read about that and went for
FOR /f "delims=" %%i IN ("!helper!") DO SET "result=%%~nxi"
still doesn't work.
Now I could really need some help and would really appreciate it :)
regards Phoenix
Next code snippet could work. Note that the SET "band=%result: - =" & SET "song=%" trick can't be performed using ! delayed expansion unlike %-expansion. Therefore, that command is moved to the :myset subroutine and executed via call command.
#echo off
SETLOCAL EnableExtensions EnableDelayedExpansion
SET "source=g:\music\folder"
FOR %%f IN (%source%\*.mp3) DO (
echo %%f
FOR /f "delims=" %%i IN ("%%f") DO SET "result=%%~nxi"
call :myset
IF not exist "%source%\!band!" MD "%source%\!band!"
MOVE /-Y "%source%\!result!" "%source%\!band!"
)
goto :skipMyset
:myset
SET "band=%result: - =" & SET "song=%"
goto :eof
:skipMyset
pause
Resources (required reading):
(command reference) An A-Z Index of the Windows CMD command line
(additional particularities) Windows CMD Shell Command Line Syntax
(special page) EnableDelayedExpansion

batch: storing folderpaths in variables

Win7/64bit/batch/beginner
To sum this up, I'm trying to
search an/all external drive/s for a certain folder and
copy it to another location.
I looked up for-loops and the xcopy stuff but I'm completely new to batch scripting so this is what I got so far:
#echo off
::I found this to be useful for variables in for-loops
Setlocal EnableDelayedExpansion
set odrive=%odrive:~0,2%
set backupcmd=xcopy /S /C /G /D /E /I /R /Y
::this part is meant to find all available drives
for %%i in (D E F G H I J) do if exist %%i: (
::and store the drive name
set target=!target!%%i:\
echo "!target!"
::now I want to find a certain folder and use its filepath
for /d /r "!target!" %%a in (*) do if /i "%%~nxa"=="foldername"
set folderpath=!folderpath!%%a ::I want to use this just like I did earlier
echo "!folderpath!" ::but it turns out to be empty
::the last step is to use the folderpath for the backupcmd
%backupcmd% "!folderpath!" "%drive%\backedUpFiles"
::reset variables
set "target="
set "folderpath="
)
No idea if any of this is actually "good" batch scripting but so far I can follow and understand it. I'm expecting the error to be somewhere between the lines (literally) and maybe it has just to do with how I'm using variables.
The error is something about "invalid drive name".
I hope someone can help me since I'm really done searching code after some days.
Try this:
#echo off
Setlocal EnableDelayedExpansion
set searchitem=level 3
set target=%drive%\backedUpFiles
set backupcmd=xcopy /S /C /G /D /E /I /R /Y
set backupcmd=echo
for %%X in (C D E F G H I J) do if exist %%X:\ (
if exist "%%X:\%searchitem%\" %backupcmd% "%%X:\%searchitem%\" "%target%"
for /f "delims=" %%D in ('dir /b/s/ad "%%X:\%searchitem%" 2^>nul') do %backupcmd% "%%D" "%target%"
)
As examples I've set the foldername to search for to "level 3" to demonstrate foldernames with embedded spaces. In the loop we've got to handle firstlevel subfolders separately (a quirk of dir, not listing the foldername but it's content). I found dir to be much faster than a for /R loop, which I didn't get to run without errors anyway.

Batch file copying script giving errors on for loop

I'm in the middle of writing a batch script that will copy a bunch of tiny files from one location to another, verify that the file exists in the new directory, delete the original, and move on. I've implemented a FOR loop to do this, but I can't seem to get it working. Any ideas?
::Copying files-
::For loop copies file, verifies file was copied,
::and removes file from source.
for %%f in (Q:%source%\*) do (
:recopy
copy %%f %dest%\%%f
if not exist %dest%\%%f goto recopy
set /a count+=1
del %source%\%%f
echo copied !count! of %total%
ping -n 1 -w 500 1.1.1.1 >nul
)
The inputs for the variables are as follows:
set /P source=Enter path to files on P:(Temp) drive (EX: jdoe\copy)
set dest=D:Temp\Copy
Scripted adding if network share is as follows:
net use Q: \\SERVER\Temp
The error given is:
The system cannot find the file specified
and the error is repeated until the screen fills.
Full pastebin at http://pastebin.com/yjXP7H6U
You can't have a label within a for loop or if statement if memory serves. You should move :recopy outside the loop. Something like this:
::Copying files-
::For loop copies file, verifies file was copied,
::and removes file from source.
for %%I in (Q:\%source%\*) do (
call :recopy "%%~fI" "%dest%\%%~nxI"
set /a count+=1
del "%%~fI"
echo copied !count! of %total%
ping -n 1 -w 500 1.1.1.1 >nul
)
goto :EOF
:recopy source dest
copy "%~1" "%~2"
if not exist "%~2" call :recopy "%~1" "%~2"
goto :EOF

Copying files based on date modified in batch file

I am trying to copy files from one directory to another using a DOS batch script. The files I want to copy are the 4 or 3 latest files. That number will be static, but yet to be determined. Is there anyway to copy based on date modified?
Thank you
You could:
1) have the dir command sort files in the descending order of date modified;
2) use the output of the dir command in a `for loop to copy the corresponding files;
3) count to 3 (or 4) in the for loop to limit the number of files copied.
#ECHO OFF
SET "srcdir=D:\Source"
SET "tgtdir=D:\Target"
SET /A topcnt=3
SET /A cnt=0
FOR /F "tokens=*" %%F IN ('DIR /A-D /OD /TW /B "%srcdir%"') DO (
SET /A cnt+=1
SETLOCAL EnableDelayedExpansion
IF !cnt! GTR !topcnt! (ENDLOCAL & GOTO :EOF)
ENDLOCAL
COPY "%srcdir%\%%F" "%tgtdir%"
)

Variables in loops Dont Work, Batch

This is the new Script and it Still Doesn't Work
I Get The syntax of the command is incorrect.
on FOR /F "USEBACKQ tokens=*" %%A IN (TYPE "C:\Windows\System32\tasks\at!num! ^| FIND "Command") DO (
SETLOCAL ENABLEDELAYEDEXPANSION
set num=1
:START
IF NOT EXIST "C:\Windows\System32\tasks\at%num%" (GOTO:EOF)
FOR /F "USEBACKQ tokens=*" %%A IN (`TYPE "C:\Windows\System32\tasks\at!num! ^| FIND "Command"`) DO (
set var=%%A
ECHO %var%
SET /a num=%num%+1
PAUSE
)
GOTO:START
To understand your code, I'm going to break it down into logic first then try to solve it. Let me know if I miss a detail...
Set num var to 0
Begin :Loop
set num var to its current value ::NOT NEEDED - You've specified this prior to the GOTO
increment num var by +1
if myfolder\at* file exists then read at%num% and find a string then output that line to %tmp%\1.txt ::Need quotations on file location.
set F var to the line stored in %tmp%\1.txt
set F="%%F: =%%" ::Please explain what you are trying to do with this command.
set F to start on 10th character and remove the last 11 characters from the line.
echo the variable
If it doesn't exist, exit, but if it does return to :Loop
You should tell us what you are attempting. If it is as simple as saving a variable from a text file output, set F=<file.txt will work. If it didn't, then something happened prior to that command. Still... what is set F="%%F: =%%"?
Unless you are using a FOR loop variable, there is no need to use %% on each end of the variable.
If this were a FOR loop, it would look like this:
SETLOCAL ENABLEDELAYEDEXPANSION
set num=1
:START
IF NOT EXIST "myFolder\at%num%.txt" (GOTO:EOF)
FOR /F "USEBACKQ tokens=*" %%A IN (`TYPE "myFolder\at%num%.txt" ^| FIND /i "string"`) DO (
PAUSE
SET var=%%A
ECHO !var!
PAUSE
SET var=!var: =!
ECHO !var!
PAUSE
SET var=!var:~10,-11!
ECHO !var!
PAUSE
SET /a num=!num!+1
ECHO !num!
PAUSE
)
GOTO:START
One good practice to check if commands are working, such as SET, insert an ECHO on the variable and a PAUSE right after each time you believe the variable should be changed. This will track what has changed on the variable so you can see if your command was correct and the changes were made.
I'd suggest using Batch's inbuilt function for loops, see here.
Conditionally perform a command for a range of numbers
Syntax
FOR /L %%parameter IN (start,step,end) DO command
Or maybe iterating over files in a folder would be better for what you are trying to do?
Loop through files (Recurse subfolders)
Syntax
FOR /R [[drive:]path] %%parameter IN (set) DO command
Or iterating over file contents?
Loop command: against a set of files - conditionally perform
a command against each item.
Syntax
FOR /F ["options"] %%parameter IN (filenameset) DO command
FOR /F ["options"] %%parameter IN ("Text string to process") DO command
This site has plenty of examples here which should point you in the right direction.
There are a few issues with your code, I've amended as follows to get the variable populated with the contents of the temp file.
set num=0
:Loop
set /a num=%num%+1
if exist "myFolder\at*" (
TYPE "myFolder\at%num%" | FINDSTR "\<Command\>" > "%temp%\1.txt"
set /P F=<"%TEMP%\1.txt"
Echo %F%
Pause
)
I don't know if this is the problem, but have you tried enabling:
setlocal enabledelayedexpansion
Then, inside the loop (or the IF(...)), you use !foo! to signify environment variables instead of %foo%.
See setlocal /? and set /? for more information.