Month: May 2021

Good Morning from my Robotics Lab! This is Shadow_8472, and today, I am working with my father, Leo_8472, on creating a master archive of our family photo collection. Let’s get started!

A Project in Motion

I’m starting this post off with a section written by Leo as my first-ever guest writer, since he has been doing most of the scanning.

Hi. Leo_8472 here, and today I am writing about my experience learning curve on using the scanning system set up by Shadow_8472 for our family photo archive. This has been a long process getting set up and now we are starting up production scanning. Shadow_8472 showed me the basics of XSANE scanning software, but here is where I go solo.

The first thing XSANE does when starting is to look for scanning devices. So, the flatbed scanner needs to already be on and fully booted up, or else XSANE will not find it. My next battle was to turn on the ‘Acquire Preview’ so that I can scan only the area of the flatbed with my photo, not the entire glass.

I eventually found the Acquire Preview Window option, but not before going down several rabbit holes. One of these rabbit holes holes pulled me in when I clicked on a gamma adjustment control on the XSANE menu and the gamma controls exploded, making the menu extend off of the bottom on the screen. XSANE would not let me move the menu up higher to reach any controls at the bottom of the menu, like the scan button. So in an effort to get more screen real estate out of the garage-salvaged VGA monitor, I swapped on an HD LCD monitor from my usual computer. Ahhh, an HD image at last. Almost. The image Button Mash was sending to my HD LCD monitor was exactly the same as what was sent to the VGA monitor. ARRGH! To get higher resolution I had to dive into the operating system’s monitor settings and raise the resolution to maximum for my new monitor. This mostly worked and I was able to see more of the XSANE menu, but not all of it.

Shadow_8472 came to my rescue to help me resolve the menu difficulties. He did not know the solution off hand, but started the process of looking up instructions for the gamma controls on the internet, ans we found a “Candelabra” toggle in the XSANE menu which will shrink the menu to a civilized size.

With the menus tamed and the Acquire Preview window showing, it is time to scan. I select a photographic print from the archive and examine the front and back of the print. Some of the photos have valuable information written on the back of the photo. (Old style Metadata.) I can often decode some of the info on the back of the photo such as a date or a name and sometimes the information is in Russian, which gives me a hard time. Anyway, photos with writing on the back get scanned both front and back. We decided to add the letter “F” for “front” and “B” for “back” to the end of the file name so that we can keep the files together.

I scan the front of the photo first, save it to a descriptive directory and then scan the back of the photo. We started scanning small photos at 1600 dpi and found the result was huge and filled an HD monitor display. 1600 dpi will pick up the texture of the surface of the photographic paper, so there is plenty of latitude for future cropping, if anyone wants to make an enlargement of this photo in the future.

The scanning process is repetitive, so I try to get into a rhythm of the steps required. Saving the scan to our Network Attached Storage (NAS) was taking at least as long as scanning and making for longer wait times. To shorten the wait, we repaired a length of CAT6 cable with a new end connector and put the scanning system on a hard wire cable rather then relying on WIFI.

We performed a test to determine our improvement in network speed by scanning a sample snapshot and timing the save using WIFI and using the CAT6 cable. We found that our sample snapshot of about 70 MB took about 34.5 seconds to save using WIFI and a second scan of 90 MB took about 4.5 seconds to save using the CAT6 cable. Saving 30 seconds per scan for this sized file is a great improvement as it is almost 10 times faster using the hard wire connection.

Future potential improvements to the scanning process would be to implement our ideas on the physical handling of the prints that we are scanning. Another important thing for this scanning project is to try to make progress every day to keep the momentum going. Eventually we will get through the whole archive.

Ethernet Enabled

Shadow back here. A while back, I was given the remnant from a spool leftover from when someone ran a network wire under our church. Originally, I was going to use it for a model supercomputer, but that project is on hold pending a better understanding of packet routing; I don’t want the individual nodes seeing other computers on the home network.

I had an idea while originally brainstorming the setup to run that cable from the router to the room where we’re set up for scanning, preferably before we cut it up into a bunch of little patch cords. We learned how to put the connectors on, but the cable didn’t work.

We eventually got confirmation from a continuity tester we ordered that a couple wires were switched, and one didn’t connect at all. In an effort to speed up saving each individual picture, we pulled out the tools to redo the ends. Leo managed to find the exact video for our crimping kit, and when the first one was done, I went ahead and tested continuity again. By chance we had fixed the bad end, and the results matched a known good cable.

I ran the cable from the Button Mash workstation directly to the router. Once I had it adjusted and everything, it had maybe a few inches to spare. We ran two tests: a ping test, and a speed test. When pinging the router at 10.0.0.1, we were hearing back about four times faster. The speed test had Leo scanning a picture and saving it once over Ethernet, and once over Wi-Fi. Ethernet finished saving in about 3 seconds. When we went for Wi-Fi, XSANE crashed while switching over, so we rescanned and saved a similar file, and it took around 30 seconds. My Pi 400 is stepping down from Wi-Fi duty.

AI Enhancement

And now for something I spent a while on, but have yet to get working. I thought it would be funny to pull a fast one on Leo by using an AI covered in one of Two Minute Papers’ YouTube videos last year. He explained how researchers came up with a new way to colorize black and white photos that addressed many issues with missing data, citing subsurface scattering where light bounces around within a subject’s skin before coming back out.

I managed to find the GitHub from the project and clone it. I tripped during configuration, not understanding the dependencies had a step to install them using an environment tool called Conda until someone pointed it out to me. Conda was a bit of trouble in and of itself. I don’t know exactly how I got it working, but one time I repeated an instruction and got a working result when expecting another failed one.

I set up the environment and switched to it, but eventually ran into an error I’m not able to blitz my way through: CUDA out of memory. All four gigabytes of my GPU (Graphics Processing Unit or graphics card) were gobbled up by the test pictures, and the model kept asking for more.

I even went to the lengths of asking my sister if I could put my hard drive in her computer to borrow her graphics card, but the thing ate through all 8 GB of hers and burped out the same error without seemingly making any additional progress.

My trouble is I don’t know what kind of hardware I need for this. I’m imagining a dedicated server with four top of the line GPU’s running headless (no monitors, accessed over SSH). There’s still the chance it could be looking at my card and thinking it can take everything and make up the difference with relatively slower resources elsewhere on the system. In that case, I’d be pouting at me over having to share with my GUI (Graphical User Interface).

Takeaway

You don’t need to wait to start a project until conditions are perfect, otherwise things will never start as new ideas are half-developed and never tested. Think small thoughts starting out as you continue dreaming big. Invest slowly as you need it.

Final Question

What sort of features would you like to see if/when I see about a site redesign?

Good Morning from my Robotics Lab! This is Shadow_8472, and today is officially –no questions asked– Day 1 of actual scanning my family’s photo archive. Let’s get started!

Scanner in Motion

We’re scanning! We’re actually, finally scanning! It feels worth almost nothing that most of my research on fancy techniques is in the bit bucket. It’s been a long year for me, yet when I picked it up, my father had had it for quite a bit longer.

It came to my attention that my main hangup is names. I don’t know who most of these people are, where they are, or when they were there. Meanwhile my father keeps doing show and tell. In the end, this is still his project just as much as it has been mine. I have researched and assembled the equipment and software, but he is going to be the main force behind putting names to everything.

Our System

We are taking some advice we found about grouping photos by location and approximate year. Batches are now defined by these groupings in their own directories, for example: LosAngeles-1952. We’ll also have a metadata file we should be able to do something with at a later time if we need it.

While the microfiber gloves aren’t working out for us to wear, they’re still microfiber, and they can still clean dust. Even then, there’s only so much that can realistically be done. Rips, scratches, and mold have destroyed original data.

Scanning is unfortunately picture by picture. There is a possibility I might try something with the GIMP XSANE plugin later, but for now, bare XSANE works well when used as intended: preview, select scan area, scan. We’re dumping our scans directly onto the NAS (Network Attached Storage), changing the name of XSANE’s preview as we save it to denote front or back when necessary.

Scanning at 1600 dpi, a postage stamp displayed at full resolution would nicely fill the old, boxy, VGA tube monitor I pulled in from the garage; a postcard natively fills a more conventional HD monitor. I was dreaming it might go up to 4K or 8K; technology progresses, but the archive will not.

To come is digital editing for the best photos. Acidic paper slowly eats itself away, giving old photos that characteristic yellow. Rips, scratches, mold, and dust will take time to remove, but it’s perfectly doable with a brush. And of course, edited photos will get a manual deskew and crop when needed, which is most of the time as I can’t stand to lose detail off the scan area, and it doesn’t line up with the glass perfectly.

Takeaway

Writing on the back is what really killed most of my grand plans. XSANE really didn’t want to accommodate in that department, my hack confused my mother when my work in progress confused her, and I never got around to touching the script I saw mention of for putting front and back side by side in the same file. On top of that, scanning backs is prone to mismatching when using the script Divide Scanned Images.

This project is a team effort now. As noted earlier, I’m being the tech support here, and my father is moving to be the star of the show. We’ll be trying for at least a package of pictures per day, and if it all goes well, we’ll bring the second workstation into the mix.

Final Question

Have you ever worked on a project, only to have someone seemingly take over, but realized it was turning into more of a group project with distinct roles?