Emergency Power Reserves

Good Morning from my Robotics Lab! This is Shadow_8472, and today I am adding an uninterruptible power supply to ButtonMash instead of actually finishing the Bitwarden job because SSL certificates are very confusing to research. Let’s get started!

Computers 101

Here is the first lesson in computer literacy: power in, stuff happens. But unlike simple light bulbs, computers are essentially endlessly shifting labyrinths of electronic circuits. Complex patterns of information flash back and forth at near light speed both within any given computer. Networked computers form even more complex circuits. Everything is timed on the scale of nanoseconds or tighter. Power a local node off suddenly, and these patterns can be interrupted. That is why it’s important to use proper shutdown procedures.

But power outages are almost always a surprise. Planned maintenance or safety shutoffs can happen, but more often than not, the power company is just as surprised as its customers left in the dark. Backup systems are about as varied as their applications. I’ve heard of an old diesel locomotive engine rigged to jump start power generation for an entire college campus. Combustion-powered generators are popular at many scales. The hardware I’m dealing with today is an Uninterruptible Power Supply (UPS) from CyberPower.

First Impressions

This particular unit is essentially a battery with an inverter and some electronics to monitor everything. Going off just the box and literature it shipped with, it can power a little network device for several hours or a gaming console and TV for half an hour. Just don’t go trying to vacuum with it or run a space heater off it.

The unit has a USB-B connector in back, but little more is said about it than it carries data. I had to do a bunch of background research online to know beyond the shadow of a doubt that my particular model here could communicate a power outage so the system can gracefully shut down. Otherwise, it’s useless for long and unattended outages.

I did not find the CyberPower website helpful. The Linux driver section kept listing RHEL/CentOS 7 and ButtonMash is running a RHEL 8 compatible system. By the time I got around to seeking personal assistance finding the right driver, we had lost the receipt and didn’t know the exact date of purchase, which was the only field labeled required by the help form.

I borrowed a phone and called in. After explaining my situation, retrospect has me believe I was shepherded into the part of the help script pointing me past the one spot where the RHEL driver included version 8, and by then I had forgotten I was still wondering if my specific model could actually use the driver. Assuming it could, it wasn’t clear if the driver could hook into Cockpit or if it would want an open port all to itself.

Open Source Driver: NUT

CyberPower’s official support may be official, but it left a bad impression. When I saw that the driver needed root privileges, I decided to research an open source driver. Not many appear to exist. I reached out to my usual Linux community support channel, but it was quiet all day. I eventually landed on a mention of Network UPS Tools (NUT).

Unfortunately, my research here is incomplete. My impression is that they’ve been around for a while and that I will need to familiarize myself with some less beginner-friendly Linux rituals to operate effectively there. NUT appears to be exclusively distributed as source code signed with GPG. I haven’t mastered either skill. Furthermore, even if I do get the driver working, my specific model isn’t on their known working list. In any case, I will need to reach out to this specific community at some point if I want to see it working.

Side Project

My sister is into a game called Stardew Valley. It recently had a slightly clunky update that broke her modded setup through something known as SMAPI. My father and I took turns working on it. I opted to try a clean install on the virtual machine I’m trying to set up for her, but I ran into a problem when the virtual drive filled up its physical space without dynamically expanding. I had made a copy of the whole VM, so I was able to delete the troubled clone. In the future, I should take snapshots to restore.

Meanwhile, my father worked on the original Windows install. Together, we traced the problem to the wrong specific version the dotnet 5 framework being used when I took the shasum of the install file and compared it against the files available for download. We later identified the problem as a failure to communicate on the part of the download page. The three Windows versions looked like they could be one download each for Windows, Linux, and Mac where the correct version was located in the “column” below the Linux tab’s heading.

Takeaway

It’s safe to say I’m well beyond beginner-friendly Linux territory now. I start doing sysadmin level stunts, I best expect to subside on sysadmin grade assistance. For now, I’m happy knowing ButtonMash can survive a brief power blink and that I have a while to manually shut it down if I notice an outage as it starts.

Final Question

Has quality of help ever chased you away from a particular product?

Slow and Steady Builds a Server

Good Morning from my Robotics Lab! This is Shadow_8472 and today I am continuing the grind on my Rocky Linux server running on my ButtonMash workstation. Let’s get started!

Quality of Life

I’ve had a single-focus mindset of getting Bitwarden/Vaultwarden working, but I’ve overlooked an important aspect of running a reliable server. One evening, any new connections to ButtonMash were failing. When I investigated the next day, the system was on and the network was connected, but when I logged in to console and pulled up the address with ip a, I found the router had incremented the IP by 1.

The task of setting up static IP’s is not alien to me. On some systems, I’ve even edited config files by hand. Cockpit has a Networking tab, though. From there, I was able to quickly worm my way into the correct network interface and configure a static IPv4 back to where it belongs. Cockpit tried to reestablish its connection with the new configuration and predictably failed. I had to make the change permanent with a follow up prompt.

Container Isolation

One of the draws of Podman over Docker is rootless containers. In the unlikely event someone hacks into Vaultwarden’s container and breaks out into the host system, he won’t be able to do as much damage if he finds himself in a non-root account – even better is if he finds himself in an account without sudo privileges.

With that, I added a user special for running Vaultwarden with sudo useradd vaultwardenUsr and I locked it with sudo usermod -L vaultwardenUsr to block any normal login attempts. sudo su vaultwardenUsr will still let me Substitute User into the account. And of course I find the tools to do all that under Cockpit tab Accounts.

From within my dedicated Vaultwarden container user, I looked for the two containers listed by the plugin. But I found nothing. From my background research I should have put together, containers are owned by users like any normal file. The two images the plugin listing their owner as “system” were actually owned and operated by root, as clarified by a test where I stepped back from Administrative access to download a new container image. Novel as it is, the plugin won’t be of much help for my use case unless Cockpit lets me access it without a password.

Bringing Vaultwarden Online

podman run -d --name vaultwarden -v /home/vaultwardenUsr/vw-data/:/data/:Z -p 44300:443 vaultwarden/server:latest

This is a long, complex command that took me while to puzzle into working order.

podman run

This is the base command. I used it to go from a container image to container.

-d

“Detach” runs containers in the background.

--name vaultwarden

By default, containers are given random pronounceable names. You can just spin up multiple copies of the same image without needing to give it much thought. Here, it’s best to have something descriptive.

-v /home/[username]/vw-data/:/data/:Z

“Volume” mounts a directory from the host into a directory in the container. This part deviates from most Vaultwarden guides the farthest. First of all, Docker installation assumes you’ll be mounting directory /vw-data/ directly from the root of your file system. However, that directory is off limits without elevated privileges, so it needs a place in the user’s home directory and a full path to that directory.

The other solved issue involves the :Z at the end. At one point, I seemingly had everything else sorted, but when ran **podman ps** to list running containers, it kept coming back empty. My online searches for Podman and Vaultwarden kept sending me back to last week’s progress-report of a post, but I eventually located a discussion on Vaultwarden’s GitHub. I didn’t do much follow up research after I got it working, but it appears to be related to SELinux (Security Enhanced Linux), a kernel level security module.

-p 44300:443

“Port” forwards ports from the host machine to individual containers: host on the left and container on the right. Recommended behavior (with Docker in mind) is to forward port 80 (HTTP), but I’d rather use HTTPS on port 443. Additionally: ports 0-1023 are reserved for system use, so I added a couple zeros on the host end.

vaultwarden/server:latest

With all the flags out of the way, this is the container image being copied into an operable format.

With the container running, all I had to do was punch a hole in the firewall and I can now have Firefox laugh at me for a failed secure connection. During assembly, I was able to make a non-secure HTTP connection with -p 8000:80 and bring up a Bitwarden login.

The next step is to obtain a security certificate, but that’s a topic for another week…

Side Project

When I walked my father through installing Debian/LXDE on his computer, he was left with a few specialty keys on the keyboard going unresponsive – particularly the volume up, down, and mute keys. The system is usable without them, but it would be nice if these keys worked. Originally, I assumed the problem would be somewhere with Debian itself, but when I tried a search targeting LXDE instead, I knew from the search results that I had today’s side project.

As we researched, I started building a system flow chart from key press to volume adjustment. The Keyboard transmits its key events to a USB receiver. The receiver communicates with some sort of keyboard driver be it discrete or compiled into the Linux kernel. The kernel then passes key events to the Xorg server. From here, we used xev (X EVent) to confirm that volume key events were all making it thus far.

At some point, PuleseAudio takes over and tells the Linux kernel what to push to the speakers and how loud to play it. While testing, we cut off a song with pulseaudio –stop / –start and had to refresh the webpage to get it back.

I personally think we’ll find the problem either within Xorg or between on a direct link between Xorg and Pulseaudio. In the meantime, the search continues.

Takeaway

For both these projects: ready-made solutions out there we could just copy, paste, and wish for luck but we’re trying to learn the innards of Linux, so it’s okay to take extra time.

Final Question

Have you ever had to program your keyboard events by hand?

Linux 101 with Leo_8472: Part 2: Browser

Good Morning from my Robotics Lab! This is Shadow_8472, and today I am hardening Firefox with my father on his new Debian installation. Once again, I am hands off the actual machine. Let’s get started!

The popular Chrome browser by Google is literally made by a company that makes money by studying you to feed you ads you’re vulnerable to (ie: more likely to click on) or want you to see for some other reason (a political agenda they approve of). While this is shady and dishonorable, respectively, there’s little to keep them from aiding a government they wish to appease from targeting people on a basis for any sort of “wrong” thinking.

Consider: if logging in to a site online is like showing your face, the browser you use to visit that site is your car. Just as people can learn to recognize you by your car, websites can recognize you by your browser –even if you don’t log in– by all the individually nondescript details your browser shares by default.

Firefox markets itself as a privacy-respecting browser, and while they’re a less-offensive choice than the alternative, they’ve made some choices that chase cash and not all their defaults respect Grandpa Joe’s technical ignorance. Their default search engine is Google. They have a “privacy respecting” news headline service that tracks you locally on your machine (as I understand it) but ultimately will record your clicks and is subject to its own political bias.

Project Progress Review

Before diving into the main payload, I did guide my father through installing Debian again over his first install – this time using the LXDE desktop environment. It’s small, and we are planning on installing a different one anyway that isn’t included in the default lineup found in the installer.

When starting a software project is often best to check for updates. sudo apt-get update and my father’s account wasn’t in the sudoers group. I quickly found a command to fix that and talked him through logging in as root to take care of that. Once updated, we began working over Firefox using a guide by Chris Xiao [1].

Firefox Hardening

Firefox presents its user with an overwhelming number of options. Xiao’s guide [1] does a good job of maneuvering you through a number of options you may want to set if present, but in the end, it’s up to you to make the final call on each setting exposed to you. We neither followed all the steps suggested nor limited ourselves to it.

The guide starts off with the settings menu one might find by fiddling with the menu, where it was simple enough to read the guide and switch settings as we spotted them. It then followed in to the spooky scary deeper settings you only change when you’re okay with potential software breaks. My father compared it to the registry in Windows.

Closing thought:

I made a mistake. In a coming week, I plan on moving my father over to the “testing” branch of Debian. I don’t have the exact numbers in front of me, but his Firefox version is about 15 to 20 versions behind what I’m using on Manjaro (Firefox 92.0.1).

I will need to revisit this list.

Takeaway

Online privacy is largely a fleeting fantasy these days, but I believe it’s still worth working for.

Final Question

Where do you draw the line between privacy, functionality, and security?

Work Cited

[1] C. Xiao, “Yet Another Firefox Hardening Guide,” May 5, 2021 [Online]. Available: https://chrisx.xyz/blog/yet-another-firefox-hardening-guide/ [Accessed Sept. 26, 2021]

By Continuing To Stay I Agree To What?!

Good Morning from my Robotics Lab! This is Shadow_8472, and today, I’ve been thinking long and hard about my policies towards current practices of data collection and how I’m going to balance privacy and functionality. Let’s get started!

Wakeup Reactions

My first reaction when thinking about data collection is that it’s all bad – overreaching companies look at where you’re going and what you’re doing and use that data to feed you ads and filter your social media feeds so you start thinking “correctly” according to some big boss at the top. I’m scared. Some people just give up trying because they feel it’s too late to stop the cyberpunk dystopia we’re quickly sinking into so it’s not worth it to even try. Others will go overdrive the other direction and redesign their whole lifestyle around having little to no online interaction.

I’d describe myself as somewhere in the middle. I’ve tried to take a reasonable stance; I only replace digital services with less invasive alternatives when they’re otherwise up for replacement. This strict of a policy has led me to go a year without a phone and to boycott Minecraft ever since they announced mandatory Microsoft accounts. Overall, I feel a little safer for my efforts, but I’d feel better if I could convince the rest of my family to take similar steps as they are able. At the same time, I feel a little more isolated than I should be.

Data Collection: the Good and the Bad

The truth about online data collection is that there are functions that logically won’t work if it doesn’t happen at all. Posting to a comments section implies permission to display said comment publicly. It may also subject it to moderation, and inclusion in site backups (it may show up again if site is rolled back to before user-end deletion). Sharing a photo may additionally require giving the service permission to “digitally manipulate” it to make a low resolution thumbnail. Ordering up a video from a streaming service requires it to know what you’re watching. All these things are fine, even though you are technically forgoing elements of privacy to allow things to work.

The bulk of applications of data collection exist in a huge gray zone between acceptable and unacceptable. That video streaming service can make more appealing suggestions by building up a profile on you, observe that all your peers are into MoneyBuck’s Capitalist Island Adventure let’s plays (not a real game so far as I can tell), and guess that you might be too! The same technology and permissions can be used to track your interest in candidates for the Yellow-flag political party and with the way things are set up, it only takes a few lines of fine print you will never see, unless you actively look for them and they can sell their profile on you to some other company who prefers you vote Orange-flag and –with another line or two of fine text you won’t likely ever see or understand– will slowly change what it shows you to make you more sympathetic to the Orange-flag party while it hides pro-Yellow-flag stuff from people already in the Orange-flag camp.

In extreme, real-life cases, they’re essentially digitally harvesting your organs to profile you for profit in a not-illegal black market. Their model of you is perfectly legal because you technically give consent to the fine print of a web page they’ll only serve up when asked specialy.

Struggles of the Privacy Resistance Movement

Privacy tools exist. Many people give up hope anyway presumably because their go-to information sites often have built their business models around the user data market – places with a vested interest in hiding tools designed to assist in undermining abusive data collection policies.

Some jurisdictions have passed laws requiring opt-out options. It’s a start, but in my experience, there’s still a lot of room for sites and services to avoid unintentionally informing users of their rights and/or make it annoying or inconvenient for users wishing to exercise them. More than once, I’ve seen an opt-out popup revert on page refresh. For user accounts, expect instructions on how to find the opt-out controls to be buried behind at least a few layers of “Please see our X for more information” worth of of tiny type.

Minecraft and Microsoft Accounts

I don’t think of Microsoft in a positive light anymore. Their announcement trailer for account migration was honestly more than a little offensive because it left me with the message that my concerns in particular could be bribed away with a cape. They can burn my cape for all I care. It wouldn’t look good with my player skin anyway. I’ve been vocal about rejecting the terms of service for a Microsoft account.

However, I’d be lying if I said I didn’t miss playing Minecraft. If I want to keep playing, there will come a time when I’ll need to migrate. I’m still not happy about getting an account, but how much of a privacy backslide is it really? What steps can I take to tell them, “I said, ‘No!’” in a way they will understand?

I reached out to the folks at tos;dr [Terms of Service; Didn’t Read]. Their whole thing is annotating terms of service documents and highlighting potentially objectionable or commendable phrases they find embedded in the legalese. But they only have so many resources. In the end, they’re good to sound the alarm and not so good for pressing forward. I wasn’t able to get anything conclusive straight from them.

As migration propaganda was pushed through my family’s Minecraft launchers, I managed to talk my family into slogging through the tiny type. A lot of it sounded pretty good. Sugary even. Alarmingly calm. One document led to another led to three more and the web went on. It was not a fun read. Only as we were coming to the end of a document chain did we find that this Privacy Dashboard they mentioned several times [1] was effectively a partial red herring; I’ll need to talk to a human being somewhere in their system to exercise my full rights. {Writer’s note: I really wanted a direct quote, but it’s late, and “Learn more” is the enemy of “Find on Page.” Consequently, I don’t know if I’m on the right page.} With all the tricks I’ve seen so far, I half-expect their staffed privacy department to be severely understaffed to the point where it barely exists, legally speaking.

I may not agree with everything I’ve seen, but there’s nothing stopping me from minimizing my contact with them. I’ll only make an account once it’s absolutely necessary. I’ll only ever use Linux when playing because it doesn’t have all the mysterious collection nodes requiring additional policies on data collection from Windows, Mac, and some other names I don’t recognize or remember. The official launcher with its ads for their other games will only be my last-resort option after days of trying to get a chosen 3rd party launcher working. Finally, I’ll lock down my account once it’s made with their Privacy Dashboard for what that’s worth while I at least try to go the final step and clamp it down hard through a live agent. If I am going to accept their terms of service, it will be on MY terms.

Takeaway

Too few people care about data collection practices and the people who do are burdened with the alarmingly rational feeling of Big Tech watching them, even if they make a hobby or career out of researching how to be as nondescript online as they might be while walking a dog. Privacy should not be something Grandpa Joe needs to worry about after little Jimmy the computer nerd tips him off. On the other hand, it’s easy to get overenthusiastic about reclaiming privacy and cut yourself off socially instead. Each person has his or her own balance point, but it’s all too often privacy by intervention instead of by default.

Final Question

Have you ever boycotted anything to seemingly no avail?

Work Cited

[1] Microsoft, “Microsoft Privacy Statement” Microsoft, July 2021.[Online]. Available: https://privacy.microsoft.com/en-us/privacystatement [Accessed Sept. 6, 2021]

Removing Backgrounds With GIMP

Good Morning from my Robotics Lab! This is Shadow_8472, and today I am editing pictures for my church’s photo directory. Let’s get started!

My Background

I first learned the basics of photo manipulation in high school using Photoshop. Pictures can be split apart, remixed with other pictures or original (hand drawn) elements with a wide variety of tools, and sent back out for use elsewhere. The whole operation is organized in a series of layers with their won operations.

Photoshop is not a bad program, but ever since Adobe moved their product to a subscription model, service has have improved for people who use their products consistantly. But for people like me who only need it once every few years and at a moment’s notice, monthly or yearly payments are a terrible investment. I own an old copy of Photoshop, but even then the liscence is a pain to move from computer to computer — and if memory serves, it’I’m limited to the total number of computers it can ever rest on.

GIMP (GNU Image Manipulation Program) is a comparable, free and open source alternative. And in true Linux fashion, it focuses on a small core feature set to serve the vast majority of users’ needs while specialized functionality can be added with extensions. Contrast that with Photoshop loading every tool imaginable for everyone who won’t use but a quarter of them at most. In short: Photoshop has more polish, but GIMP is easier on system resources.

Selecting and Removing a Background

My church produces a photo directory every few years. Families grow, and pictures need updating. I was given a number of pictures and was expected to remove their backgrounds or else turn them white. Photoshop’s tool of choice would have been the quick select tool, but GIMP won’t have a perfect counterpart for this simple tool until GIMP 3.0. Instead, I taught myself the foreground select tool, which I would describe as resistant to learning by trial and error.

The foreground select tool has multiple phases. In GIMP 2.10 specifically, you start with a selection loop, overestimating your subject. You can use line segments or hand drawn boarders, whichever you deem most reasonable at the time. Endpoints are freely adjustable once you’ve closed the loop.

Hitting Enter will put you in phase 2. Here, the whole picture takes on a blue tint (The color is adjustable, but I didn’t learn how). The goal here is to paint in your clear foreground, heavily tinted background, and a lightly tinted buffer between them called “unknown” or “unknown pixels.” Tool options let you switch between what zone you’re painting.

Here, more than anywhere else, it’s particularly picky about keeping on task. I lost about two hours total because I did something just a little too unrelated and the foreground select tool sent me back to phase 1 with no way to undo. It is best to save other tasks in the workflow for later, like adding an alpha (transparency color) channel or duplicating/hiding the original picture layer for quick reference.

Phase 3 will “[calculate] the alpha value of unknown pixels” referring to your boarder zone. It provides a preview of exactly what you’re selecting and lets you make further adjustments, but it will recalculate everything each paintbrush stroke you make. I recommend finding the small dialogue box that lets you disable the preview, effectively reverting to the more economical phase 2. By going back and forth between these two phases, you can tell where the algorithm understands your selection and where it needs more help. Hitting one enter while in phase 3 will finalize your selection.

Training Family

One of the more rewarding parts of this experience was teaching my father this tool, though there were unique challenges. At one point, he was having frustrations getting the tool to cooperate. I had worked with this tool using two or three computers, but his description of what was happening sounded alien to my experiences.

I took a look at his workflow, and GIMP was skipping straight from what I’m calling phase 1 to phase 3, forcing him to use long strokes or face tons of extra recalculation time. Long story short, while I was trying to make his tool icons larger, I found he was on GIMP 2.8, the latest available in the Linux Mint repositories. The GIMP download page directed me to a GIMP 2.10 flatpack installer. The system still prefers GIMP 2.8 when unspecified, but he figured out how to proceed, so that’s a tech support topic for another day.

Color to Black and White

Later on, the software for the directory was making some pictures way too dark. Photoshop has a layer filter for this exact use case with six sliders to adjust different color ranges. GIMP doesn’t include any layer filters, but I was able to make a crude approximation of this tool. Just know there may be a better way to do this and I don’t know it — plugin or otherwise.

To make the perfect black and white version of a color picture, I would start by adding a white background and merging layers if necessary. Next, I menued over to an option to decompose the color channels to a new document. With each layer in grayscale, I added another white background and used Color to Alpha on the red, green, and blue color channel layers. I then adjusted the opacity of each layer until I had a result I was happy with.

Each picture was its own challenge. Most times, I would have one layer carrying most of the data while reducing the other two dramatically. Too little color data, and the picture would fade. Too much, and it would go dark. I targeted a mix where the picture would pop and my eye would relax a little while looking at it.

Takeaway

Both GIMP and Photoshop are wonderful pieces of software in their own rights. I’ve had a class on Photoshop, so of course I’m going to find it easier to navigate that program, even if it’s been years. Despite my slim requirements of advanced photo editing, GIMP’s slim feature set slimmed out a feature or two I miss.

Final Question

Have you taught yourself both Photoshop and GIMP? Which is harder to learn?

Attempting a Personal Wiki

Good Morning from my Robotics Lab! This is Shadow_8472, and today I am trying to set up a personal wiki to understand my own site better. Let’s get started!

Searching for a Wiki

Pick a game, show, or fandom, and chances are you will find a wiki for it — sometimes more. These online subject encyclopedias are normally open for readers to edit, and otherwise need no introduction.

I already knew there existed at least one version of the software free and open source, but Wikipedia has a whole list of them [1]. The table was static [note: only on mobile], so I had to mentally track for process of elimination across categories like open source vs proprietary or even its target audience.

My use case is to make a wiki about my sister’s fanfiction series. She’s built up her own take on the world of Sonic, and we decided it could be fun to learn how to organize elements of her story into a familiar format. If successful, the same process will make it easy for us to grow similar wikis from other stories, like when we’re worldbuilding for a possible role play game.

I landed on pursuing Foswiki [2]. It’s not limited to a single user, it’s free and open source, and it has most or all the features tracked in the comparison cited above. I did not notice at the time, but it expects you to work with Perl, a programming language of which I know little more than the name. I am open to learning, though.

Foswiki Host

For the record, I currently lack a dedicated home server I have complete control over while ButtonMash is on picture scanning duty and my fleet of Pi’s is dealing with unknown issues: either a poorly chosen brand of SD cards or one of the machines going bad. Besides, I was hoping some of you finding this post might be interested in doing this at home, so I wanted to use one of my workstations.

I landed with using DerpyChips. Even though it’s not always the most stable, it is the most readily accessible of my workstations from any point on the home network. I encountered some trouble updating though: a repository in its listings was no longer signed. It wasn’t until the next day of work that I noticed it was just a repository for Celestia, a computerized home planetarium. So far, the issue has been ignored, but it ate up a lot of the time I otherwise would have used towards the main project. In the meantime, I had reached out to three or four places, including the PopOS Matermost chat and the Celestia Discord once I identified the problem.

Apache2

A web site cannot run on bare operating system alone, apparently. There’s an extra layer called a web server. Apache Web Server is a name I’ve heard in relation to the web for a while. I gather it’s a popular choice. It was also mentioned in the installation documentation for Foswiki, and only after I installed it by repository did I notice that it’s only one of a few supported options.

Apache installed smoothly. sudo apt-get install apache2 and Firefox was able to reach the test page for Ubuntu at Derpy’s IP.

My only reservation at this point is a possible mix up between Apache and Apache2. It’s just a version difference, but I have not spotted Apache2 being called out in Foswiki’s documentation.

Takeaway

I fully hoped this would be a one and done project. Once understood and scooted the problem to the side, Foswiki appears more involved than I expected. I don’t want this to become another long-term project, so I may take another look at that list again and see if there’s something geared a little more towards a brother and sister organizing thoughts.

I find it odd that Foswiki doesn’t capitalize their W when most other wikis appear to.

Final Question

How would you use a personal wiki to organize information?

Works Cited

[1] “Comparison of Wiki Software,” Wikipedia. Accessed: June 13, 2021. [Online]. Available: https://en.wikipedia.org/wiki/Comparison_of_wiki_software

[2] Foswiki. Accessed: June 14, 2021. [Online] Available: https://foswiki.org/

Family Photo Chest: Part 15.1: Incremental Improvement

Good Morning from my Robotics Lab! This is Shadow_8472, and today, I am working with my father, Leo_8472, on creating a master archive of our family photo collection. Let’s get started!

A Project in Motion

I’m starting this post off with a section written by Leo as my first-ever guest writer, since he has been doing most of the scanning.

Hi. Leo_8472 here, and today I am writing about my experience learning curve on using the scanning system set up by Shadow_8472 for our family photo archive. This has been a long process getting set up and now we are starting up production scanning. Shadow_8472 showed me the basics of XSANE scanning software, but here is where I go solo.

The first thing XSANE does when starting is to look for scanning devices. So, the flatbed scanner needs to already be on and fully booted up, or else XSANE will not find it. My next battle was to turn on the ‘Acquire Preview’ so that I can scan only the area of the flatbed with my photo, not the entire glass.

I eventually found the Acquire Preview Window option, but not before going down several rabbit holes. One of these rabbit holes holes pulled me in when I clicked on a gamma adjustment control on the XSANE menu and the gamma controls exploded, making the menu extend off of the bottom on the screen. XSANE would not let me move the menu up higher to reach any controls at the bottom of the menu, like the scan button. So in an effort to get more screen real estate out of the garage-salvaged VGA monitor, I swapped on an HD LCD monitor from my usual computer. Ahhh, an HD image at last. Almost. The image Button Mash was sending to my HD LCD monitor was exactly the same as what was sent to the VGA monitor. ARRGH! To get higher resolution I had to dive into the operating system’s monitor settings and raise the resolution to maximum for my new monitor. This mostly worked and I was able to see more of the XSANE menu, but not all of it.

Shadow_8472 came to my rescue to help me resolve the menu difficulties. He did not know the solution off hand, but started the process of looking up instructions for the gamma controls on the internet, ans we found a “Candelabra” toggle in the XSANE menu which will shrink the menu to a civilized size.

With the menus tamed and the Acquire Preview window showing, it is time to scan. I select a photographic print from the archive and examine the front and back of the print. Some of the photos have valuable information written on the back of the photo. (Old style Metadata.) I can often decode some of the info on the back of the photo such as a date or a name and sometimes the information is in Russian, which gives me a hard time. Anyway, photos with writing on the back get scanned both front and back. We decided to add the letter “F” for “front” and “B” for “back” to the end of the file name so that we can keep the files together.

I scan the front of the photo first, save it to a descriptive directory and then scan the back of the photo. We started scanning small photos at 1600 dpi and found the result was huge and filled an HD monitor display. 1600 dpi will pick up the texture of the surface of the photographic paper, so there is plenty of latitude for future cropping, if anyone wants to make an enlargement of this photo in the future.

The scanning process is repetitive, so I try to get into a rhythm of the steps required. Saving the scan to our Network Attached Storage (NAS) was taking at least as long as scanning and making for longer wait times. To shorten the wait, we repaired a length of CAT6 cable with a new end connector and put the scanning system on a hard wire cable rather then relying on WIFI.

We performed a test to determine our improvement in network speed by scanning a sample snapshot and timing the save using WIFI and using the CAT6 cable. We found that our sample snapshot of about 70 MB took about 34.5 seconds to save using WIFI and a second scan of 90 MB took about 4.5 seconds to save using the CAT6 cable. Saving 30 seconds per scan for this sized file is a great improvement as it is almost 10 times faster using the hard wire connection.

Future potential improvements to the scanning process would be to implement our ideas on the physical handling of the prints that we are scanning. Another important thing for this scanning project is to try to make progress every day to keep the momentum going. Eventually we will get through the whole archive.

Ethernet Enabled

Shadow back here. A while back, I was given the remnant from a spool leftover from when someone ran a network wire under our church. Originally, I was going to use it for a model supercomputer, but that project is on hold pending a better understanding of packet routing; I don’t want the individual nodes seeing other computers on the home network.

I had an idea while originally brainstorming the setup to run that cable from the router to the room where we’re set up for scanning, preferably before we cut it up into a bunch of little patch cords. We learned how to put the connectors on, but the cable didn’t work.

We eventually got confirmation from a continuity tester we ordered that a couple wires were switched, and one didn’t connect at all. In an effort to speed up saving each individual picture, we pulled out the tools to redo the ends. Leo managed to find the exact video for our crimping kit, and when the first one was done, I went ahead and tested continuity again. By chance we had fixed the bad end, and the results matched a known good cable.

I ran the cable from the Button Mash workstation directly to the router. Once I had it adjusted and everything, it had maybe a few inches to spare. We ran two tests: a ping test, and a speed test. When pinging the router at 10.0.0.1, we were hearing back about four times faster. The speed test had Leo scanning a picture and saving it once over Ethernet, and once over Wi-Fi. Ethernet finished saving in about 3 seconds. When we went for Wi-Fi, XSANE crashed while switching over, so we rescanned and saved a similar file, and it took around 30 seconds. My Pi 400 is stepping down from Wi-Fi duty.

AI Enhancement

And now for something I spent a while on, but have yet to get working. I thought it would be funny to pull a fast one on Leo by using an AI covered in one of Two Minute Papers’ YouTube videos last year. He explained how researchers came up with a new way to colorize black and white photos that addressed many issues with missing data, citing subsurface scattering where light bounces around within a subject’s skin before coming back out.

I managed to find the GitHub from the project and clone it. I tripped during configuration, not understanding the dependencies had a step to install them using an environment tool called Conda until someone pointed it out to me. Conda was a bit of trouble in and of itself. I don’t know exactly how I got it working, but one time I repeated an instruction and got a working result when expecting another failed one.

I set up the environment and switched to it, but eventually ran into an error I’m not able to blitz my way through: CUDA out of memory. All four gigabytes of my GPU (Graphics Processing Unit or graphics card) were gobbled up by the test pictures, and the model kept asking for more.

I even went to the lengths of asking my sister if I could put my hard drive in her computer to borrow her graphics card, but the thing ate through all 8 GB of hers and burped out the same error without seemingly making any additional progress.

My trouble is I don’t know what kind of hardware I need for this. I’m imagining a dedicated server with four top of the line GPU’s running headless (no monitors, accessed over SSH). There’s still the chance it could be looking at my card and thinking it can take everything and make up the difference with relatively slower resources elsewhere on the system. In that case, I’d be pouting at me over having to share with my GUI (Graphical User Interface).

Takeaway

You don’t need to wait to start a project until conditions are perfect, otherwise things will never start as new ideas are half-developed and never tested. Think small thoughts starting out as you continue dreaming big. Invest slowly as you need it.

Final Question

What sort of features would you like to see if/when I see about a site redesign?

Family Photo Chest: Part 15: Day 1, for Real This Time

Good Morning from my Robotics Lab! This is Shadow_8472, and today is officially –no questions asked– Day 1 of actual scanning my family’s photo archive. Let’s get started!

Scanner in Motion

We’re scanning! We’re actually, finally scanning! It feels worth almost nothing that most of my research on fancy techniques is in the bit bucket. It’s been a long year for me, yet when I picked it up, my father had had it for quite a bit longer.

It came to my attention that my main hangup is names. I don’t know who most of these people are, where they are, or when they were there. Meanwhile my father keeps doing show and tell. In the end, this is still his project just as much as it has been mine. I have researched and assembled the equipment and software, but he is going to be the main force behind putting names to everything.

Our System

We are taking some advice we found about grouping photos by location and approximate year. Batches are now defined by these groupings in their own directories, for example: LosAngeles-1952. We’ll also have a metadata file we should be able to do something with at a later time if we need it.

While the microfiber gloves aren’t working out for us to wear, they’re still microfiber, and they can still clean dust. Even then, there’s only so much that can realistically be done. Rips, scratches, and mold have destroyed original data.

Scanning is unfortunately picture by picture. There is a possibility I might try something with the GIMP XSANE plugin later, but for now, bare XSANE works well when used as intended: preview, select scan area, scan. We’re dumping our scans directly onto the NAS (Network Attached Storage), changing the name of XSANE’s preview as we save it to denote front or back when necessary.

Scanning at 1600 dpi, a postage stamp displayed at full resolution would nicely fill the old, boxy, VGA tube monitor I pulled in from the garage; a postcard natively fills a more conventional HD monitor. I was dreaming it might go up to 4K or 8K; technology progresses, but the archive will not.

To come is digital editing for the best photos. Acidic paper slowly eats itself away, giving old photos that characteristic yellow. Rips, scratches, mold, and dust will take time to remove, but it’s perfectly doable with a brush. And of course, edited photos will get a manual deskew and crop when needed, which is most of the time as I can’t stand to lose detail off the scan area, and it doesn’t line up with the glass perfectly.

Takeaway

Writing on the back is what really killed most of my grand plans. XSANE really didn’t want to accommodate in that department, my hack confused my mother when my work in progress confused her, and I never got around to touching the script I saw mention of for putting front and back side by side in the same file. On top of that, scanning backs is prone to mismatching when using the script Divide Scanned Images.

This project is a team effort now. As noted earlier, I’m being the tech support here, and my father is moving to be the star of the show. We’ll be trying for at least a package of pictures per day, and if it all goes well, we’ll bring the second workstation into the mix.

Final Question

Have you ever worked on a project, only to have someone seemingly take over, but realized it was turning into more of a group project with distinct roles?

I Got Git, But No Pi

Good Morning from my Robotics Lab! This is Shadow_8472, and today, I am starting out with Git. Let’s get started!

Previously…

Last week, I was all set to install Git on BlinkyPie, my Raspberry Pi 3B+, but it had some problems with accessing the apt store. I felt lucky getting away with barely fixing what I broke, but the cause remains unidentified.

Git

Disclaimer: this will not be a comprehensive tutorial starting out. There are plenty of those already. If you’re just starting out, I recommend a shorter tutorial. I viewed three videos ranging from 15 minutes to an hour (I still haven’t finished the hour long one) and found the information density roughly the same with longer tutorials covering features and configurations I’m guessing aren’t important for a single user first project. Focus on the main loop, then expand. For any one who missed my last post, GitHub is a site hosting the most well known and widely used Git server. I won’t be touching it today.

Every time I do any research about Git, it seems about ten times bigger then last I remembered. It’s free, it’s open source, and it’s even industry standard for tracking changes in a project (Version Control). I should have been using it ten years ago, but like pretty much any tool originally written with Unix or Linux in mind, its initial impressions when using it on Windows were that of an aquarium at a dog shelter; it’s been made to work, but it hardly looks like it belongs. I strongly recommend project or two in a Unix like environment to learn the command line if you want to become fluent in Git without relying on a graphical interface.

Git Server Installed, Not Configured

I have yet to see what is of BlinkyPie, but I was easily able to install Git server onto GoldenOakLibry, the Network Attached Storage (NAS), from repositories over the web interface. Had this been restricted, like what happened with my Bitwarden server, I would have proceeded with rigging a Pi to host while storing the data proper on the NAS.

Proper server side configuration will be a matter for another week. My goal was to have a test repository accessible from all my workstations. However, it appears I need to do such things as make a git user and set up credentials and user keys. I’m not prepared to do that this week.

A Simple Start

I know it doesn’t sound like much, but I have taken my first steps toward intuitively understanding Git. I made a directory and used git init to tell Git it is a repository. Git in turn made a hidden administrative directory called .git, which I don’t touch on pain of messing up the project history. I made a simple text file and walked myself through adding it, and committing it with some difficulty following on-screen instructions when it wanted a note about the commit.

The next step would have been to push it to a repository I’m at a loss for setting up. While writing, I went through the same steps again on GoldenOakLibry with no idea how I’m supposed to use a server except possibly using clone on all my client machines. I even went through the confusion that is vim terminal text editor, since nano wasn’t around and won’t be for the time being.

Takeaway

Git is big. Every programmer should learn it. Just don’t go learning both Git and the command line at the same time. I had trouble learning which was which way back when. And if you reasonably can, install Linux. All it takes is a small distribution on a thumb drive; an old computer is even better.

Final Question

Got Git yet?

Family Photo Chest Part 14.4: Assembling a Scanning System

pricingGood Morning from my Robotics Lab! This is Shadow_8472, and today, I’ve studied another photo scanning system available on the web. Let’s get started!

Introduction

Scanning seems evermore a topic I’m learning about without entering production. I’ve lost count of how many workflow iterations I’ve come up with, yet each time I feel closer than ever to that most elusive batch #0001.

I’ve now studied the free portions of two workflows from people who claim years of experience: scanyourentirelife.com and howtoscan.ca, but howtoscan is my favorite between the two. Both give enough of their respective systems away to be functional, but they each offer additional, step by step, click by click training affordable to anyone rich enough to afford a dedicated scanner as is recommended when scanning thousands of photos at home.

My outline today is based on what I’ve gleaned from freely available sources, though influences of the above two are more prominent in my mind.

1. Have a System

Anyone can throw a few prints in a scanner and share them online. Hundreds or thousands of pictures taken and organized by people spanning generations mandates some form of structure.

My system at this point looks like it may be sorting pictures by immediate family of origin and further sorting by year. Eventually, each item will end up in one of several 10-12 gallon buckets that can be scanned and moved to a neat “completed” stack.

2. Select Hardware Wisely

You don’t need a $50,000 professional scanner if you don’t know how to use the $150 one intended for home use you may already have lying around, like me.

If, on the other hand you are buying a scanner new and already have scanner software piked out, like XSANE, you should make an effort find a fully compatible model. Remember: marketing for home grade scanners doesn’t always have quality results in mind; they’re there to sell a product to the general public. This probably isn’t you if you’re taking the time to research.

Thought should also be given to storage. You may choose to store your photos on a cloud or locally, but if you’re planning your digital archive lasting another 3 to 5 generations, I would recommend some form of protection against hardware failure, such as a RAID configuration featuring redundancy.

3. Assemble a Workstation

Less important is the workstation you’ll be scanning at. In my experience so far, RAM is a limiting factor. Right now, I’m working with a keyboard, mouse, and monitor are straight out of the 90’s. Other than that, your average workstation should be fine.

I will note that since I have animals in the house, I have a room with a door to keep them out. I will also note that I feel safe enough leaving the door open when I’m in the room with all the pictures in the buckets mentioned above.

4. Capture Natural Scans and Fix in Post

Again, standalone scanners aimed at households will have modes invented by the marketing department to sell scanners. Scan at your final resolution, dust and all. Anything you do to a scan preview is permanent to your scan proper. Take advantage of automatic numbering in names. Work on a copy in a program designed to manipulate images, not one intended to sell scanners. Scan in batches with similar settings to save time fiddling with software settings.

My setup uses GIMP. I’ve taken a class on the basics in Photoshop, and about the only thing I miss is something like the Quick Select tool, but GIMP beta has an early version out for testing before their big 3.0 release. I also plan on using a script called DivideScannedImages so I can scan multiple pictures in a single pass.

Now this last part will hurt, but not everyone will be interested in his or her 8th grade school photos or the backside of a motorcycle frame. Pay the most attention to the pictures people will want to see.

5. Manage Your File System

Whatever file system you learn, learn it. Metadata is your friend. Learn how to add tags, dates — for all I know you can add whole pictures (would be nice for photo backs annotated in foreign scripts, but probably not). HowToScan recommends learning how to use batch handling, and the Linux command line usually has it baked in.

Takeaway

I can see why people might spend years figuring this stuff out and valuing it at X hundred dollars to sell the specifics. Some people need step-by-step, click-by-click instructions, but I don’t feel the need for me. I had a class in Photoshop, and most of the important points in one of the howtoscan free ebooks was all about using the “healing brush,” a technique I once used to remove a power line from a church building photo — the line art of which is still used in the directory to this day.

This has been a hard push of a month. I expect to return to assorted topics next week.

Final Question

How much would you pay to learn how to scan?