{"id":1296,"date":"2023-10-16T12:00:00","date_gmt":"2023-10-16T19:00:00","guid":{"rendered":"https:\/\/letsbuildroboticswithshadow8472.com\/?p=1296"},"modified":"2023-10-16T02:37:56","modified_gmt":"2023-10-16T09:37:56","slug":"rocky-server-stack-deep-dive-2023-part-1","status":"publish","type":"post","link":"https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/2023\/10\/16\/rocky-server-stack-deep-dive-2023-part-1\/","title":{"rendered":"Rocky Server Stack Deep Dive: 2023 Part 1"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Good Morning from my Robotics Lab! This is Shadow_8472 and today I am revisiting my Vaultwarden server to better understand the process. Let\u2019s get started!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Monday Evening (October 9, 2023)<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">OK, so here\u2019s the deal from memory: I want to put Bitwarden on my tablet. My Bitwarden server is a 3rd party implementation called Vaultwarden. Official Bitwarden clients, such as the one I want to put on my Android tablet, make use of the same encryption scheme as your average browser that relies on a system of certificates going back to a root authority. I made my own authority to get my project working, so my computers will complain by default about the connection being insecure. My desktops have been content warning me every so often, but Android is fresh ground to cover. This past week, I noticed an expired certificate, so it\u2019s time for maintenance.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It\u2019s early in the week, but this has been a historically difficult topic. I\u2019ll be using my December 13, 2021 post as a guide [1].<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Monday Night<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">I\u2019ve assessed this project\u2019s state: \u201cdusty.\u201d Vaultwarden is containerized on ButtonMash in a locked down account configured to allow-lingering (non-root users otherwise get kicked eventually). The last time I touched Vaultwarden was before I learned the virtue of scripting as opposed to copy-pasting complex commands I will need again in months to years. I foresee the next topic in my Relearning Linux series.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">My goal for tomorrow is to brush up on my terminology and generate a new certificate authority.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Tuesday Afternoon<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">While reviewing my above-mentioned post, I remembered Caddy Web Server and looked it up. Its self-signed security certificate functionality was a little much hassle for me to work out around June 6, 2022 [2]. I shelved it pending a better router before further experimenting with Let\u2019s Encrypt for real certificates, which still hasn\u2019t happened.<\/p>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-8f761849 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:66.66%\">\n<p class=\"wp-block-paragraph\">Also in my June 6, 2022 post, I owed my disappointment to difficulty with the official Caddy tutorials assuming loopback access vs. me running on a headless server being left without clear instructions to find certificate I need to install into my browser on another computer. One idea coming to mind would be to find a browser container to VNC into. But then I\u2019d still be left without the automatic certificate unless I made my own container with Caddy+Firefox+VNC server. An excellent plan Z if I ever heard one because I am not looking to explore that right now, but it isn\u2019t not an option.<\/p>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:33.33%\">\n<p class=\"wp-block-paragraph\">VNC: Virtual Network Client, Remote desktop. A desktop environment hosted one place, but rendered and operated from across a network.<\/p>\n<\/div>\n<\/div>\n\n\n\n<p class=\"wp-block-paragraph\">For a plan A, I\u2019d like to try the documentation again. Perhaps the documentation will be more clear to my slightly more experienced eyes. For what it\u2019s worth, Caddy is still installed from before.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Tuesday Night<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">I\u2019ve explored my old Caddy installation, and my limited understanding is starting to return. My big snag was thinking I needed to use the default HTTP ports while running Caddy rootless. Subdomains may have been involved. I\u2019m interested in trying Caddy containerized, where I have a better grasp on managing ports. Furthermore, if domain names are of concern, I believe involving Pi-Hole may be my best bet, and with that I fear this project just became a month long or longer.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Anyway, I\u2019m going to look into Pi-Hole again. I have an instance up and passing domain names, but I\u2019ve not gotten it to filter anything. Hopefully interpreting a host name won\u2019t be too difficult a goal for tomorrow.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In the meantime, I noticed PiHole was complaining about a newer version available. I went to work on it, but the container was configured to be extremely stubborn about staying up. I force-deleted the image, and PiHole redownloaded using the updated container. I\u2019ve left a note with instructions where I expect to find it next time I need to update.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Wednesday Afternoon<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">To refresh myself, my current goal is get PiHole to direct buttonmash.lan to Caddy on a underprivileged port for use with reverse proxy access within ButtonMash.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I found an unambiguous tutorial affirming my general intuitions with Pi-Hole itself, meaning my problems are with directing requests to Pi-Hole. For this I went to my router, which wanted an update from July, but had trouble downloading. I hit refresh and it found a September update. It installed, and I had to manually reconnect a Wi-Fi connection (OpenWRT on a Raspberry Pi).<\/p>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-8f761849 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:66.66%\">\n<p class=\"wp-block-paragraph\">I\u2019ve blown a good chunk of my afternoon with zero obvious forward progress since the last paragraph. Where to start? I\u2019ve learned <strong>nslookup<\/strong>, a tool for testing DNS. I validated that my issue with Pi-Hole is getting traffic to Pi-Hole with it. The real temptation to unleash a horde of profanities was my home router from Tp-Link. The moment I pointed DNS at ButtonMash on 192.168.0.X: \u201cTo avoid IP conflict with the front-end-device, your router\u2019s IP address has been changed to 192.168.1.1 . Do you want to continue to visit 192.168.0.1?\u201d That was a chore to undo. I needed a browser on a static IP \u2013 meaning my RedLaptop sitting in retirement as a server.<\/p>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:33.33%\">\n<p class=\"wp-block-paragraph\">DNS: Domain Name Service. This is the part of the Internet that translates the domain names within URL&#8217;s into IP address so your browser can find a requested web page.<\/p>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-8f761849 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:100%\">\n<p class=\"wp-block-paragraph\">I\u2019ve appealed to r\/pihole for help [3].<\/p>\n<\/div>\n<\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Wednesday Night<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">I\u2019ve gotten a few bites on Reddit, but my problem isn\u2019t solved yet. I performed an update on my laptop and installed similar tool to nslookup (Debian didn\u2019t come with it) to verify that OpenWRT wasn\u2019t the culprit. I\u2019ve found forum posts<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">No forward progress today, but I did make some \u201csideways progress.\u201d At least I knew how to fix the netquake I caused.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">My goal for tomorrow: I don\u2019t know. IPv6 has been suggested. I\u2019ll explore that. Maybe I\u2019ll move my OpenWRT router for my upstairs workstation over to assign ButtonMash\u2019s Pi-Hole.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Thursday Afternoon<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Good day! Late last night I posted my appeal to r\/pihole to r\/techsupport\u2019s Discord. While answering the responses I received last night, I ran my nslookup on a 192.168.0.X workstation receiving an IP over DHCP \u2013 and I got an unexpected success. Silly me had run my test over my statically addressed laptop. Also, leaving it overnight gave DHCP a chance to refresh. The test still failed upstairs workstation, so pointed OpenWRT at ButtonMash instead of the router for DNS. I\u2019ll still need to cleanup my static IP\u2019s, but I should be good to play with pointing PiHole at Caddy.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">My next big task is configuring Caddy.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Thursday Night<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">My next big task is preparing Vaultwarden for use with Caddy.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Vaultwarden is one of the longest-running and most important services on ButtonMash. It was for Vaultwarden I started learning Podman. It was for Vaultwarden I learned how to allow-lingering on a server account. It was for Vaultwarden I researched NGINX and Let\u2019sEncrypt to serve as a reverse proxy and HTTPS support, but later found Caddy to do the same things with hopefully less hassle. Vaultwarden has been with me so long, it was Bitwarden-rs (or some variant thereof) when I started using it and I didn\u2019t follow the name when it was changed. With Vaultwarden, I\u2019ve grown from dumping a raw command into a terminal to using a script on into turning that script into a rootless systemctl service that will refuse to stay down until stopped properly.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">But problematically, Vaultwarden holds port 44300, which I want for Caddy. It\u2019s also been running with an integrated HTTPS support module called ROCKET which the developers strongly discourage for regular deployments. This security certificate\u2019s expiration is what is driving me to make progress like mad this week. I\u2019ve spent the past couple (few?) years learning the building blocks to support Vaultwarden properly along with other services to run along side it. It\u2019s time to grow once again.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I spent an hour or so re-researching how to set Podman up as a systemd service. And then I found a note pointing me towards the exact webpage that helped me conquer it last time. With that, I feel the need for a break and take Friday off. I\u2019ll be back after Sabbath (Saturday night) or on Sunday to write a Takeaway and Final Question. I\u2019ve got to give all this information some time to settle.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Of note: I did come across a Redhat blog post about an auto scale-down feature not present on RHEL 8, but it looks interesting for when I migrate to a newer version of Rocky Linux in seven months when active support ends. <a href=\"https:\/\/www.redhat.com\/en\/blog\/painless-services-implementing-serverless-rootless-podman-and-systemd\">https:\/\/www.redhat.com\/en\/blog\/painless-services-implementing-serverless-rootless-podman-and-systemd<\/a><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Sunday Morning<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">This has been a very successful week, but I still have a way to go before I finish my server[\u2019s software] stack: Nextcloud on Android. I researched a couple possible next steps as a possible encore. Caddy would be the next logical step, but Bitwarden has the port I want it listening on. Alternatively: I dug up the name \u201cUnbound,\u201d a self-hosted DNS server to increase privacy from Internet Service Providers, but I\u2019d want more research time.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Takeaway<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Again: what a week! I think I have my work lined up for the month. As of Sunday night, my goal for next week is Caddy. With Caddy in place, I\u2019ll have an easier time adding new services while punching fewer holes in ButtonMash server\u2019s firewall.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Final Question<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Even now, I\u2019m questioning if I want to migrate Rocky 9 or hold out and hope Rocky 10 shows up before\/around Rocky 8\u2019s end of life. Rocky\u2019s upstream, Red Hat Enterprise Linux (RHEL) doesn\u2019t have a clockwork release cycle like Ubuntu, and RHEL 8 appears to be planned for support much longer than Rocky 8. That feature I learned about does sound tempting. Upgrade or hold out?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I look forward to hearing from you in the comments below or on my <a rel=\"noreferrer noopener\" href=\"https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/discord-invite\/\" target=\"_blank\">Socials<\/a>!<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Works Cited<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">[1] Shadow_8472, \u201cSelf-Signed Vaultwarden Breakdown,\u201d Dec. 13, 2021. [Online]. Available: <a href=\"https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/2021\/12\/13\/self-signed-vaultwarden-breakdown\/\">https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/2021\/12\/13\/self-signed-vaultwarden-breakdown\/<\/a>. [Accessed Oct. 16, 2023].<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">[2] Shadow_8472, \u201cI Switched My Operations to Caddy Web Server,\u201d June 6, 2022. [Online]. Available: <a href=\"https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/2022\/06\/06\/i-switched-my-operations-to-caddy-web-server\/\">https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/2022\/06\/06\/i-switched-my-operations-to-caddy-web-server\/<\/a>. [Accessed Oct. 16, 2023].<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">[3] Shadow_8472, et. all, \u201cTp-Link isn&#8217;t sending traffic to containerized Pi-Hole (I think??),\u201d<em>reddit.com<\/em>,Oct. 11, 2023. [Online]. Available: <a href=\"https:\/\/www.reddit.com\/r\/pihole\/comments\/175wp2l\/tplink_isnt_sending_traffic_to_containerized\/\">https:\/\/www.reddit.com\/r\/pihole\/comments\/175wp2l\/tplink_isnt_sending_traffic_to_containerized\/<\/a>. [Accessed Oct. 16, 2023].<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Good Morning from my Robotics Lab! This is Shadow_8472 and today I am revisiting my Vaultwarden server to better understand the process. Let\u2019s get started! Monday Evening (October 9, 2023) OK, so here\u2019s the deal from memory: I want to put Bitwarden on my tablet. My Bitwarden server is a 3rd party implementation called Vaultwarden. &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/2023\/10\/16\/rocky-server-stack-deep-dive-2023-part-1\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Rocky Server Stack Deep Dive: 2023 Part 1&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[90],"tags":[62,127,294,293,292,295,291],"class_list":["post-1296","post","type-post","status-publish","format-standard","hentry","category-red-hat-server","tag-bitwarden","tag-caddy-server","tag-nslookup","tag-pi-hole","tag-rocky-linux-8","tag-server-stack","tag-vaultwarden"],"_links":{"self":[{"href":"https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/wp-json\/wp\/v2\/posts\/1296","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/wp-json\/wp\/v2\/comments?post=1296"}],"version-history":[{"count":3,"href":"https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/wp-json\/wp\/v2\/posts\/1296\/revisions"}],"predecessor-version":[{"id":1299,"href":"https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/wp-json\/wp\/v2\/posts\/1296\/revisions\/1299"}],"wp:attachment":[{"href":"https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/wp-json\/wp\/v2\/media?parent=1296"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/wp-json\/wp\/v2\/categories?post=1296"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/letsbuildroboticswithshadow8472.com\/index.php\/wp-json\/wp\/v2\/tags?post=1296"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}