What Queue Folders does
Queue Folders is a file system watcher for Windows platforms that associates a local folder (or "watch folder") with a queue in RPM Remote Print Manager® ("RPM") Elite. Queue Folders monitors each folder you configure, then sends the path for new files to RPM. RPM then adds the file as a job and removes the file from the watch folder.
What happens when you create a job
We tested Queue Folders by creating files in the watch folder with a program, by copying files and by moving them. We have used the DOS command prompt, Windows Explorer, and Python scripts. Whatever means you use to create a file, Queue Folders detects the new file and sends information about it to RPM. We do this by getting file events from Windows.
It turns out that Windows can send file events much faster than we can create jobs. Using diagnostic logging, I determined that my system supports 40 file creation events per millisecond. For that reason, RPM keeps the jobs in a work queue until they are all created.
You can see this in action if you want. Run the RPM UI and select a queue with a watch folder. You’ll notice the queue list at the left has columns Name, Jobs, and Total. The total column is the cumulative number of jobs you’ve sent to this queue.
Now, copy several hundred files to that watch folder. The number in the total column will continue to spin up for a time after you copy the files. On my system, it takes anywhere from 10 to 40 ms to create a job, which works out to roughly 25 to 100 jobs per second, compared to 40,000 per second that Windows can potentially send us.
Access
When RPM creates the job, it needs to have unrestricted access to the file, because it’s going to look up some information on the file; then copy it to a folder where jobs are spooled for that queue, and finally remove the file. We realize that the other program might not be done with the file so we wait for about 1 second to begin this process.
RPM will look up a username associated with the file, so it opens the file. If another process still has the file open, RPM will retry in five seconds.
On my system, Windows returns the result “Administrator” rather than “Dave” as I would expect. Windows Explorer shows me “Dave” but since I’m part of the network administrator group perhaps “Administrator” makes sense.
RPM will use the filename part of the path as the job name.
RPM creates jobs in the order they are sent to the folder.
How Queue Folders handles files already in the folder
If Queue Folders is assigned to watch a folder that already has jobs in it, it will get all the filenames, then sort in order of the “last write time”, when the file was last updated. Then the files are sent to RPM to be processed into jobs.
It’s worth noting that the “copy” and “move” commands in the command prompt might create a file in your watch folder with the same “last write time” as the file had in its original folder. Because of this, you might not get the files to process in the order you expect, and there’s nothing we can do to change that since “last write time” seemed more important than any other metric.
However, if you are generating files from program output, you can rely on the “last write time” to be the same as when the program finishes
There are two times when you would run into this:
- When you launch RPM, and Queue Folders is licensed, and you have files in one or more of the watch folders you’ve configured Queue Folders to monitor
- When you manually configure a queue with the UI, while Queue Folders is already running, to watch a folder which already has files
Where did my files go? And when?
Queue Folders notifies RPM about the files, but it doesn’t do anything with the files on its own.
Remember that RPM receives the file info from Queue Folders faster than it can create jobs. Since we don’t want to present any delays to Queue Folders, so that it can monitor files uninterrupted, RPM caches the files and creates jobs when it can.
RPM will remove the file from the watch folder as one of the last steps in creating the job. You will notice that if you send a large number of files to the watch folder at a time, they don’t disappear all at once; the number reduces over time.
And remember that RPM is attempting to process jobs in the order they are created on your hard drive, which means that it may take some from one watch folder, then from another. Again, we felt the priority was “in the order created” which may not always come out alphabetically.