menu

Posted on by Arnon Erba in Meta

It’s been about two years since my last meta post, so in keeping with tradition it would seem I’m due for another ground-up redesign of my blog and a retrospective post to go along with it. This time, though, I think I’ll skip the redesign and jump straight to the retrospective post.

I’ve written about the history of this blog before, and much of what I wrote in 2018 still holds true today. I’ve been blogging under the Arnon on Technology moniker for a while now and the past two years have seen the publication of some of my favorite posts. Still, I’ve managed to almost continuously break the cardinal rule of blogging by averaging, in many cases, less than a single post per month.

Why? To be honest, writing posts for this blog takes a lot of time. Some of the more in-depth how-to guides I’ve written have over a dozen hours invested in them and were composed over a period of weeks or months as time permitted. For example, I’ve been working on an article about Git off-and-on since February of this year, and it’s still far from complete. In some way, it’s a case of perfect being the enemy of good enough.

In the end, that’s OK, because this blog still isn’t about money or fame. I’ve never had the time or desire to pursue ad revenue and I’m not active on Twitter or other social media platforms. Instead, my main goal is to continue writing articles about poorly documented tech problems while giving back to the community of Internet bloggers who have helped me out time and again with similar niche issues.

For now, though, it’s time to finish a few more drafts.

Posted on by Arnon Erba in How-To Guides

On Windows and macOS, Stata can be configured to check for updates automatically with the set update_query command. However, there are a few drawbacks to this approach.

For one, this feature isn’t present on the Linux version of Stata. For two, this command doesn’t actually update Stata — it just enables update notifications. Stata will still need to be manually updated by someone with the permission to do so.

If you’re running Stata on a standalone Linux server or an HPC cluster, you may be interested in having Stata update itself without any user interaction. This is especially useful if Stata users do not have permission to update the software themselves, as is often the case on shared Linux systems.

We can enable true automatic updates with a cron job and a Stata batch mode hack:

0 0 * * 0 echo 'update all' | /usr/local/stata16/stata > /dev/null

Adding this line to root’s crontab will cause the update all command to be run every Sunday at 12am. Standard output is piped to /dev/null to prevent cron from sending unnecessary emails.

As always, think carefully before enabling automatic updates for mission-critical pieces of software. However, this approach can save time over updating Stata manually.

Posted on by Arnon Erba in General

Life as a sysadmin is constantly entertaining. Some days, even when you think you’ve accounted for every possible contingency, something happens that still manages to take you by surprise. Wednesday was one of those days.

I manage a few production Red Hat Enterprise Linux servers that, until Wednesday of this week, were all running RHEL 7. RHEL 7 is still well within its support window, but ever since RHEL 8 came out in May of last year I’ve been preparing to proactively upgrade my systems. By chance, I finished my preparations this week, so I scheduled an in-place rebuild of one of my less critical servers for the Wednesday the 29th.

The Upgrade Begins

Because true hardware-based RAID controllers are blisteringly expensive, I like to run simple software RAID arrays with mdadm where possible. The RHEL installer makes it easy to place all your partitions — even the EFI system partition — on an mdadm array during the installation process, so I started my server rebuild by creating a few RAID 1 arrays on the server’s dual HDDs. Later, with the installation complete, I rebooted the server and was greeted by a fresh RHEL 8 login prompt at the console.

Shortly after that, things went sideways. I ran yum update to pull down security patches and discovered a kernel/firmware update and a GRUB2 update. During the update process, I noticed that the server had slowed to a crawl, so I checked /proc/mdstat and realized that mdadm was still building the RAID 1 arrays and was eating up all the bandwidth my HDDs could muster while doing so. Impatient, and eager to get out of the loud server room and back to a desk, I decided to reboot the server to apply the kernel update so I could finish setting things up over SSH.

No Boot?

Two minutes later, I was staring at a frozen BIOS splash screen. As I’d just installed a new network card, I immediately suspected hardware problems, so I powered the server down and checked things over. Nothing helped: The hardware seemed fine, but it still wouldn’t boot.

Mdadm is pretty resilient, but since I’d shut the server down mid-verification I hastily assumed I’d somehow broken my RAID setup. Because I hadn’t gotten very far post-installation, I decided to wipe the server and reinstall RHEL 8 to rule out any issues. This time, I let mdadm sit for an hour or so before I touched anything, and then patched and rebooted the server again. Cue the frozen BIOS splash screen.

In hindsight, the common factor was clearly the updates, but as I’d just updated my RHEL 8 development server the day before with no ill effects I didn’t immediately consider a bad update as a possibility. Instead, I reset the BIOS to factory defaults and reviewed all my settings. When that didn’t help, I rummaged through my drawer of spare parts and grabbed an unused NVMe SSD to replace the server’s frustratingly slow HDDs in case they or the RAID configuration was the source of the problem. After installing RHEL 8 on the new drive, I rebooted the server several times to verify everything worked before applying updates. Once again, everything was fine until I applied the GRUB2 updates.

Verifying the Problem

Faced with what now seemed to be a bootloader issue, I went back to my RHEL 8 development server and updated it again. Sure enough, a new GRUB2 update popped up, and when I rebooted after applying it I got stuck at a black screen. Confident that I’d narrowed the issue down to a bugged update, I reinstalled RHEL 8 one last time on my production server — this time, skipping the update step — and set about reinstalling software on it.

When I finished later that night, I got the Red Hat daily digest email summarizing the latest RHEL updates. As it turned out, Red Hat had released patches for the BootHole vulnerability just a few hours before I arrived on-site in the afternoon ready to rebuild my server. (For reference, the RHEL 8 patch is RHSA-2020:3216 and the RHEL 7 patch is RHSA-2020:3217.) I quickly disabled automatic updates on the rest of my servers and wrote up a hasty bug report at 10:15pm.

The Results

I woke up on Thursday morning to 50+ email notifications from Bugzilla and a tweet from @nixcraft linking to the bug report. As the day went on, it became apparent that RHEL 7 was also affected and certain Ubuntu systems were suffering from the fallout of a similar patch.

As of the writing of this post, it seems like the specific issue lies with shim rather than GRUB2 itself. Right now, Red Hat is advising that people avoid the broken updates, and they’ve published various workarounds that may come in handy if you’ve already applied them. For the moment, I still have automatic updates disabled, and I’m hoping that Red Hat will publish fixed versions of GRUB2 and shim soon.

In the end, I spent four hours reinstalling RHEL 8, submitted a hastily written bug report, and became the anonymous “user” mentioned in the first paragraph of an Ars Technica article:

Early this morning, an urgent bug showed up at Red Hat’s bugzilla bug tracker—a user discovered that the RHSA_2020:3216 grub2 security update and RHSA-2020:3218 kernel security update rendered an RHEL 8.2 system unbootable.

Posted on by Arnon Erba in Op-Ed

Zoom’s meteoric rise to the top of the video conferencing market makes sense when you consider that their platform is, in fact, fairly good. So far, Zoom has managed to avoid many of the pain points that plague other video conferencing solutions:

  • No account is required to join a call, and calls can be joined from any platform (even from Android, iOS, and Linux).
  • Call quality is excellent, even on slow Internet connections.
  • The platform can be used for free by anyone and it’s easy to create an account.
  • Zoom is a standalone product with a singular focus rather than an add-on feature like Microsoft Teams or Google Hangouts.

Additionally, Zoom’s user experience is good, especially when it comes to joining a meeting. It’s hard to simplify that process much further than “here, click this link”. Overall, Zoom is easy enough to use, and it manages to keep its myriad of advanced features from getting in the way of hosting simple meetings.

UX Versus UI

With that said, Zoom is still far from perfect. While Zoom’s user experience (UX) is relatively frictionless, I’m still surprised by inconsistencies that exist in its user interface (UI). In my opinion, Zoom struggles with two main UI issues: consistency and clarity.

Consistency

For one, the website shares none of the design cues of the desktop and mobile apps. Buttons don’t even share the same names: the blue “Host a Meeting” link on the website competes with a large orange “New Meeting” button in the app. The meanings of both buttons are clear upon inspection, but this means that users have to become familiar with two separate interfaces before feeling comfortable using Zoom.

Some website features don’t appear in the app, and vice versa. For example, the “Previous Meetings” tab on the website doesn’t map back to anything in the “Meetings” section of the app. Similarly, the links in the navigation bar at the top of the app (Home/Chat/Meetings/Contacts) don’t match anything on the website.

Most frustratingly, the “Schedule a Meeting” interface on the website is substantially different from the one in the app. While the website prompts you to choose a start time and duration (and only allows you to adjust those times in half hour increments), the app asks for a start time and end time. The form options themselves are labeled differently: the start time is listed as “When” on the website and as “Date” in the app. At the bottom of the form, the advanced meeting options aren’t even presented in a consistent order between the two interfaces.

Finally, weird things occasionally happen while using Zoom. For example, when I went to unmute someone in a recent meeting, the participant list started arbitrarily reordering itself every few seconds. This made it almost impossible to choose the right person, as someone else would jump under my mouse cursor before I clicked. I’m still not sure if this was an intended feature, but it doesn’t make sense that it would be.

Clarity

On the clarity side, some of the buttons in the app aren’t immediately recognizable as buttons. The Join Audio/Share Screen/Invite Others trio is the most notable example of this issue:

I’ve been using Zoom for a while now, and every time I launch a meeting I have to remind myself that those three pieces of clip art are actually clickable. Additionally, since the introductory screen vanishes once other participants join, these buttons can’t be relied upon throughout the duration of a meeting.

Conclusion

Zoom’s UI does get one thing right: every button is clearly labeled with a description of what it does. While this may not be the most aesthetically pleasing choice, it makes it easy for untrained users to get started with Zoom. For that reason alone, I appreciate Zoom’s simple and unglamorous UI, but it’s still important that it isn’t so simplistic and inconsistent that it detracts from the user experience.

Further Reading

Posted on by Arnon Erba in News

Update 2/12/2020: Microsoft has reversed their decision to automatically install the Microsoft Search in Bing extension. The extension will still be made available but will not be automatically deployed with Office 365 ProPlus. The original post continues below:

Starting next month, Microsoft plans to use Office 365 ProPlus to push a browser extension for Google Chrome that will change users’ default search engines to Bing. Version 2002 of Office 365 ProPlus will forcibly install the Microsoft Search in Bing extension for all Chrome users who do not already use Bing as their default search engine.

Understandably, many system administrators are frustrated with the announcement, as unwanted browser extensions that change end-user settings are usually considered malware and are blocked accordingly. In fact, Microsoft’s own security tools already block dozens of programs that exhibit similar behavior.

On GitHub, users are responding to the change by opening issues in the OfficeDocs-DeployOffice repository. So far, it does not appear that Microsoft has responded to this influx of unsolicited feedback outside of publishing a blog post extolling the virtues of Bing.

Who Is Affected?

At this point, only businesses that have deployed Office 365 ProPlus are affected. Depending on the organization’s Office 365 license, ProPlus is the version of Office delivered to end-users when they install Office from the office.com portal. According to Microsoft, not all Office 365 plans include the ProPlus version of Office:

This extension is included only with Office 365 ProPlus. It isn’t included with Office 365 Business, which is the version of Office that comes with certain business plans, such as the Microsoft 365 Business plan and the Office 365 Business Premium plan.

Firefox Is Next

According to Microsoft, a similar extension for Firefox is also on the way:

Support for the Firefox web browser is planned for a later date. We will keep you informed about support for Firefox through the Microsoft 365 Admin Center and this article.

Removing The Extension

By making the extension an opt-out feature, Microsoft is putting the onus on system administrators to deploy a method for blocking its installation. While there are official ways to prevent the extension from being installed, there is no easy Microsoft-supported method for removing the extension once it has already been deployed. Instead, Microsoft recommends running the following command as an administrator on each affected machine using a script:

C:\Program Files (x86)\Microsoft\DefaultPackPC\MainBootStrap.exe uninstallAll

It should also be possible to blacklist the extension with the 3rd party Group Policy templates for Chrome and Firefox provided by Google and Mozilla.

Unfortunately, Group Policy and other enterprise management tools do not always apply to BYOD devices, leaving users who install Office on their personal machines with little recourse except to notice and remove the extension on their own if they find it undesirable.

Sources