“Beowulf Cluster:” Part 3: Networking Nightmare

Good Morning from my Robotics Lab! This is Shadow_8472, and today, I actually spent the better part of the week on this segment and still didn’t get as far as I would have liked. Let’s get started!

There is a reason networking scared me off when I first looked into it back when I was in university. Every computer needs a way to introduce itself to not only its communication counterpart, but all the message carriers along the way. It’s almost unbelievable how easily it all can be smoothed over so even a toddler can get him or herself into trouble without proper supervision. But all the inner workings are still there, churning away, ready for someone to go monkeying around until something breaks.

Last week, I talked about the hardware side of things. I have all the equipment I need ready to use, but it’s waiting on a digital support structure to hold the worker nodes together. My goal for this week was to build that support structure. I would have declared success when I could connect my laptop to an Ethernet network served by the new Pi 4 and access a web page.

Originally, I was aiming to set the router up as a DHCP server so it could hand out IP addresses on the fly. Along the way, I learned how to set up a static IP. Did I mention how ugly raw networking configuration looks to complete beginners?

I honestly don’t know half the stuff I did as I was trying to follow multiple tutorials at once, hoping one would fill in for another when it didn’t apply to my present situation. One blank on a text-based tutorial was elaborated on by a quick comment by a YouTube video, and I ended up stumbling in a third direction in the hopes I was doing the right thing for my project.

And that’s when I can even follow the tutorials! Half the time I landed on a video where some would-be instructor may know what he’s talking about, but his audio quality says he’s working with what he’s got, and that can be a bit sharp on the ears. Another good chunk of the time, it’s someone with a thick accent I wouldn’t be able to understand unless I could ask him to repeat multiple words. Unfortunately, text is going to be the way to go for tutorials at my level of specialization.

I wasn’t able to get the DHCP server sorted out. Something about there can only be one per network, and I have no idea how to confine it to confine it to one network interface — that’s another thing I learned about.

Network interfaces: the whole reason this project is a thing is because the Pi 4 has two separate network interfaces, or ways to access a network, such that it can participate in two distinct networks at once: one for the Ethernet, called Eth0, and one for the WiFi, called wlan0. If that weren’t confusing enough, there’s a new standard that names network interfaces off the hardware they’re connected to. For my intents and purposes, where I only have a single interface per kind of connection, it only serves to confuse when it shows up elsewhere — namely my laptop — but I imagine it would make handling a 50 port switch a little more bearable.

At some point, I gave myself a reality check. I only had five computers to contend with. DHCP is a little bit much for what I’m looking to do. Extra overhead: needs to be axed. I’ll just use static IP’s all around. I went ahead and set my laptop’s Ethernet network interface up with one and rebooted, since I couldn’t seem to restart just the network settings. I had to fix a couple settings, but it was pretty easy to get it to ping –and even ssh into– the Pi 4 over the Ethernet network I had going. The trouble was that I couldn’t access a web page like I wanted.

I messed around a bit and when I was done, I unplugged and connected to my home network, only to find it was being difficult. The best I can figure is that by editing one of the configuration files to give me a static IP, I was accidentally voiding some default for WiFi settings as well. I made an attempt to copy what I had going on from a tutorial on the Pi 4, but due to a series of disconnection issues, I ended up restarting the Pi and losing it on the network.

I was sure to make notes on how to reverse the network damage to my laptop as I made it. I commented out the altered lines, and rebooted again. Still no Pi 4 on the network. I looked on the miniature network again, and eventually resolved to swapping over the HDMI cable to the back of the monitor, only to find a line in ITS network configuration files from when I was experimenting with iptables to solve the present issue.

I’m laying this issue to rest where it is for now. I had some measure of tangible success, but the majority of my efforts were sunk into learning new skills. My laptop is in a usable state, at least.

Final Question: What have you cut out to make things easier on yourself?

Beowulf Cluster: Part 2: Not a Beowulf Cluster Yet

Good Morning from my Robotics Lab! This is Shadow_8472 and today, I learned a lot toward the next step in my project and backed out at the last minute. Let’s get started!

You know the thing about clusters: it takes some overhead to run them? The computers I’m working with in my Third Workshop care package don ‘t have much CPU to begin with. Special thanks to kevrocks67 on the EngineerMan Discord server for getting that point through my head and pointing me in the right direction: subnetting.

My understanding of subnetting is lacking to say the least. Perhaps if I back up and just tell the story.

This week saw most of the parts I need for my cluster come in. Shopping for the Pi 4, I compared different sellers and chose one to go along with a separate heat sink case I picked out from a different distributor. I managed to save a little money by assembling my own kit instead of getting one that included some small, aluminum heat sinks.

When the case arrived, it came with some thick thermal pads. Yeah, those won’t be separating once the case is fully installed and operational temperatures melt the pads into place or they are just sticky. I did a dry fit and it doesn’t look like the case will operate correctly without them. In the meantime, I noticed the camera cable slot and the similar-looking display slot will likely be inaccessible once I assemble the case, though the GPIO pins will still be open.

My father picked up a couple outdoor grade 3-way power spliters for power management. The minicomputers I’m working with have bulky transformers right on the plug, and they are such a shape as to make it difficult to creatively arrange more than three on a seven-plug power strip.

The Ethernet cable we picked up was paid for with a little effort. One of my church members was wiring up the church a while back for Cat6, and he has some left on the spool. I stayed topside while my father donned a jumpsuit and finished running the cable to just below the office, where he cut the cable. There was enough to reach the office, but that was it: maybe about 30 feet max: more than double my very rough estimation of what I’d need. A package of Cat6 ends and a crimper will see me learning yet another new skill in the near future.

About the only piece I don’t have picked out yet is a board to mount things on. It doesn’t look like the heat sink case has any good places to mount, though the switch and minicomputers do.

The next logical thing to do is assemble an early prototype! One control node and one worker node should be a good start… if only I could get the pi to boot from an SD.

Lacking the desire to format any drives where a valued drive can possibly be hit in the crossfire, paired with the inability to read SD cards on anything but my laptop, I tried using an app on my tablet to flash a card. For one reason or another, I wasn’t able to boot. I suspect I need root privileges to properly format anything with Android, unless I want to format as internal or removable storage. Lacking access to an old phone nobody has no qualms with me “voiding the warranty,” a multi-SD reader is on order.

Speaking of SD readers, turns out each of the five little helpers has one built in. I’ll be using one of those along with a live USB for formatting fun in the near future with no chance of fireworks at the end, just as soon as I figure out why they look exactly the same with lsblk -l. I suspect it’s got its own SD card like storage device in there. Ironically enough, I suppose the difference should make a slip up of the same calibur nearly impossible this go around. Nevertheless, I still want to practice isolation of formatting whenever possible. In either case, what I am trying to think of as equivalent to a hard drive is not labeled sdx anymore.

Somewhere, I made the switch in short-term project aim from cluster to subnet to reclaim some of that overhead. I still want to end up with a cluster for the experience, but for now my token effort among token efforts will be more efficient with less overhead.

To help with overhead costs, I found a site that compares lightweight distros. Thishosting.rocks/best-lightweight-linux-distros/ They have a sortable chart of about 30 distros and their rough minimum system requirements, including RAM, CPU, and space on disk. Folding@Home is likely to use CPU above anything else, I suppose, and RAM can’t be easily expanded, so my dump stat is disk space.

I optimized for CPU, and found Debian was a near-perfect choice. Most, if not all, my Linux experience is in Debian or a decedent of Debian. The required RAM is one of the lower rankings while still needing an almost non-existent CPU. I even already have the install media plugged into the thing and booted! Though I may want to look into a server installation option.

In terms of mentally tangible progress toward a prototype, I set the Pi 4 aside for the moment and brought in BlinkyPie, my Pi 3B+. My goal: turn a Raspberry Pi into a router. I’ll know everything works when I have Internet access on the minicomputer that is otherwise not connected through any other means.

I viewed a few networking videos that were either dated, covering basic material I already knew, had bad audio quality, or showed what to do without giving any theory behind their actions. I eventually settled on a clear, step by step tutorial on at Linuxhint.com/raspberry_pi_wired_router that does not explain its actions, but the steps are short enough that I can seek out my own explanations as I go.

The tutorial starts with installing Raspian and getting SSH going. As anyone who’s been following my blog long term knows, SSH was a real pain for me to figure out, and if I didn’t know what I was doing, it might as well be a small element of a magic ritual to start over if things fail. Since I intended to work with a Pi that was already established, I glazed over those steps and started in on configuring the network.

The whole reason this project of a subnet is possible with the hardware I have is because the Pi can participate in two networks at once: one via Ethernet, and one via WiFi. To make a long story short: this procedure requires the Pi establish itself in a special IP address for the router, implying a static IP. I sometimes take this Pi elsewhere (or at least its chip), and I don’t want to forget and accidentally mess up someone’s network, so I backed out in favor of waiting on a properly running Pi 4.

As always, this project is evolving as I work on it. The Gigabit Ethernet speeds I thought I would use for a cluster won’t be necessary for the time being, as there’s a slightly lower bottleneck at the Pi’s WiFi network adapter. Next week should be interesting.

Final Question: When was your last project where you lowered the specs and found yourself overbuilt?

Beowulf Cluster: Part 1: Sum of Its Parts

Good Morning from my Robotics Lab! This is Shadow_8472 and today, I am planning a model supercomputer. Let’s get started!

I feel swamped with projects right now. They end up stacking up and with different priorities, so I’m glad I can finally do something with this lockdown care package I recieved from Third Workshop several weeks ago.

The package contained five Minus Forum Fanless Mini PC’s, model N40. After carefully removing and discarding the wrapper, I pulled one out and hooked it up. It booted to Windows 10 setup. Nice that they included something, but for a number of reasons, I will be installing some version of Linux on these tiny machines. What annoyed me was that the screen where Windows wants me to accept the EULA did not have a reject button.

What annoyed me more was how I never received an answer from support about a partial refund per the Windows EULA. Honestly, I did that more on principle, and I don’t know how it would work. I am not the original purchaser, and they were already on sale. Token effort. Their support website didn’t have a security certificate for protecting login details, so I didn’t prioritize pursuing a couple dollars per machine any further. In the meantime, I turned my attention to booting Linux from my Debian install stick. Let me tell you: the BIOS on these things is about as user friendly as their tech support.

By default, it priorities internal boot devices over removal media, and it only gives a tiny window of opportunity to enter the boot menu, or it’s off to Win10. it’s almost like the original manufacturer didn’t want people looking at the BIOS at it’s almost like the original manufacturer didn’t want people looking at the BIOS at; I had to look up what key to press, and it was either F2 or DELETE, and when I got to the BIOS menu, it was the ugliest one I’ve ever seen. The aesthetics were on the low end for terminal style menus, but the real kicker was how they handled moving the boot order around. I wasn’t able to figure it out except that it felt like swapping items in an array instead of moving things up and down a list. Despite the challenges, I was able to boot the way I wanted to.

Assembling a supercomputer though, that is a whole other story. There are two kinds of supercomputers: distributed, like what Fold@Home is using to combat the CCP Virus Pandemic (which I intend to add these goodies to), and clusters, like what I’m trying to build. At present, I am pursuing the term Beowulf Cluster in my searches.

I just wanted to touch on another reason not to go with Windows for this project, besides me transitioning to Linux whenever possible. Windows charges per instance of their OS. Supercomputers need one OS per node, and practical supercomputers will often have at least five or six nodes working in parallel. Couple that with the already sparse number of applications for supercomputers, and you end up with virtually no supporting software to use Windows for a Cluster computer.

The biggest challenge I faced in this part of the work was knowing what to order. I had to ask someone on my family’s Minecraft server before I knew for sure that a switch would serve just fine. Then came the rest of the support system.

These little computers support Gigabit Ethernet, so that’s what I’m aiming for all along the system, primarily the network cabling. There’s a lot to know about wiring. Modern data transmission builds on the twisted pair technique originally invented by Alaxander Gram Bell. Since then, we’ve found ways to reduce noise, such as insulating the wires from each other, shielding them from each other with foil, shielding the whole set of wires from their environment with more foil, and so on and so forth. There are specs on fire safety if the cable is installed horizontally, vertically, or even in the ventilation system.

More relevant to my project is how the cable’s wires are made. One kind is made with lots of strands, making it easy to bend around and is often used between computers and the wall. The one I want uses solid wires; the wires won’t be moving around much after installation.

Rounding the computation side of things off, I have a Raspberry Pi 4B on order. That was a small adventure. Turns out the B in Pi XB just means it has an Ethernet port, and a model A doesn’t have it. Since they aren’t making Pi 4’s without Ethernet at present, all cases made for plain “Pi 4” should work for my purposes. I went for one that doubles as a large heat sink. Neither have arrives as of writing.

The final hardware challenge is power. While I’d like to just slap a strong enough power supply on the mounting board, different parts of my system will be running three unique voltages. The only ones I’d have any chance of combining are the original minicomputers, and their power supplies are matched pretty well with what they need. They also take up too much real estate on the power strip. I researched over and out of my price range, landing on the conclusion that more research would be needed to combine transformers. The plan now is to grab a couple angled three-way splitters and use the original transformers.

I haven’t even gotten to assembling the system, and it’s already a formidable challenge. Nevertheless, there is one more good reason for combining all these machines in one “casing:” I believe my family is near capacity with the number of computers our router can physically address at once, and that this project would otherwise put it over. The way it’s planned out, as the controller. the Pi 4 will be the only node to talk to the outside.

Final Question: What would you use a supercomputer for?

ButtonMash Upgrade Take II

Good Morning from my Robotics Lab! This is Shadow_8472, and today, I am actually doing a one-and-done project. Granted it’s more of a follow up to a huge project I did last year, but still, it felt great getting something short in. Let’s get started!

My family and a few online friends have enjoyed ButtonMash, a Minecraft server I assembled from hardware I had laying around, for several months now, and it recently ran out of hard drive space. Turns out 60 Gb of space doesn’t last all that long when you’re making full weekly backups of the whole server! I started covering this upgrade when I accidentally formatted my Windows drive on my laptop.

I started with a new download of MineOS. I formatted the correct drive and used the dd command to write the image file to the correct drive. With a new install media, I brought it and the new 1 Tb SSD to my tower and hooked them up to be the only things connected. Installation went smoothly until MineOS didn’t recognize the USB WiFi adapter I have.

I put the project down for a while while until I had a chance to continue working on it using an Ethernet connection. The only SATA port my laptop has is hidden below about twenty to thirty screws, the ButtonMash server wasn’t going down for any longer than I could help. Aside from two computers I highly doubted I could use, I was left with the one I put together for my father. With his permission, I installed the drive and ran the security updates recommended earlier during setup.

The only real snag I hit was when I ran into the DHCP configuration requiring manual intervention each time the machine booted. Annoying, as I was working with it mostly from SSH. I knew I had had this issue before, but I couldn’t remember how I solved it. I was a little frustrated with the lack of information online on this topic, but it was amusing for a previous blog post to show up in my search results — twice. My father ended up manually configuring a static IP one boot. It took another hour or two before I remembered static IP was the way to go. I added “fixing the static IP” as something to do announcing project completion.

Copying over the server files was fun. I got into the old server’s web interface and made a fresh archive. Then, using the new install of MineOS, I SSH’ed into the old one to copy the archive over to an import folder using scp. I goofed on the long file path, so I ended up copying it to the home directory and then over to the new hard drive.

With that, I moved over to the new server’s web interface and imported the server. Everything was there, except any backups. I logged into Minecraft and got in no problem, and when I checked, the automatic updates were still around. I’m thinking about redoing those as monthly, but it works for now, nobody touch it!

As a side note, I did some digging into using Let’s Encrypt with MineOS. A thread on the MineOS forums filled in some gaps in my understanding of MineOS’s https security heigine. Https relies on a security certificate –usually issued by a trusted third party– to scramble their communication. When the server self-signs a certificate, you as a client have to trust a potential unknown with keeping their half of the code safe. Before, I was under the impression there wasn’t any sort of security certificate without Let’s Encrypt. However, since the only people who need encrypted access are admins looking to access the web interface, they should probably be in a position to trust a self-signed certificate. From an outside attacker’s point of view, it looks the same as one from a more official source.

I shut both computers down and moved the hard drives around. My father’s computer was reassembled, and ButtonMash’s small MineOS SSD and Ubuntu hard disk were pulled and replaced with the new SSD. I booted it up, logged in with SSH (Public/private key authentication is so nice!), and fixed the static IP from when it had adopted my father’s computer’s IP as static. One more reboot later, and the server was officially up. Within half an hour, one of our players from the outside was on.

One more side note: it looks like by default, you have to first ssh in as a standard user, mc by default, and then use the su (substitute user) command to switch to the root account, as sudo does not come with MineOS by default.

On a finishing thought, I honestly hope someone finds this useful, even if I end up rambling. I purposefully added some key words I wish I had covered last time, so maybe I’ll come across this one next time I find myself setting up MineOS.

Final Question: Have you ever searched for help, only to be redirected back to yourself?