Author Archives: Ben Ogorek

Introducing “datascroller” for fast terminal data frame scrolling

Category : Python , Tools

I’m excited to announce my very first package on PyPi, datascroller, a Python package for interactive terminal data scrolling. It’s available for Windows as well as *nix systems (thanks to windows-curses), and contributors to the codebase are welcome!

How it works

See the gif below for a glimpse of datascroller in action:

datascroller allows terminal datascrolling

The syntax has changed slightly since the gif was created, but during that demo, I was pressing keys to resize the terminal viewing window and to scroll from left-to-right and up-to-down within a Pandas data frame. Currently the scrolling keys are inspired by vim but later versions will offer customization options.

You can install datascroller with pip using:

pip install datascroller

Try datascroller out in iPython with the following code:

import pandas as pd
from datascroller import scroll

train = pd.read_csv(
    'https://raw.githubusercontent.com/datasets/house-prices-uk/master/data/data.csv')

scroll(train)

Why a terminal data scroller?

Scrolling a through a data set is a fundamental part of exploratory data analysis, and open-source tools let us down in this regard. SAS has had it right for a while. From my memory of around 2001, you could scroll through tens of millions of rows through what must have been a very clever paging strategy. Say what you want about SAS, but honestly no other data viewer has come close.

Moving to R in 2009, I had to accept the loss of SAS’s data set viewer and learn to accept the built-in viewer or just print slices of the data frame in the console. Soon after, I started using RStudio. They offered a nice improvement on the default viewer, but it still couldn’t hold a candle to SAS’s and didn’t handle very large data sets well at the time (to the best of my recollection).

In 2019, RStudio may very well have their data viewer tuned to perfection. But some people prefer working in the terminal, and sometimes you have to (say, a client gives you an ssh login for a particular remote machine). It is possible to hook up notebooks, or use an X-server, but often it’s easier to just print slices of your data sets in the terminal for exploratory analysis. Ehile R’s tibble and Panda’s DataFrame are smart enough to not overwhelm your console with output, they make you work to see the parts of the data that you really need to see.

The datascroller vision

The featured image is a play on the movie “Minority Report” and its very memorable scene with Tom Cruise’s character using the futuristic API to sort through information. I always wanted to move around the data set like that, and I felt that the terminal would be a good place to do it. In 2014, at Google, I took my first crack at this with an internal R package I called “terminalR.” I got helpful feedback from data scientists there, especially Tim Hesterberg. Tim convinced me of the need to implement user configuration options (still a TODO for datascroller!) and also to transition to Emacs/ESS since they came with Emacs Lisp. But, we stopped short of achieving the vision full interactivity.

The terminalR package’s original mechanism was “drumming” on the enter key while you pressed other navigation buttons, as it relied on R’s standard console input methods). With Python offering wrappers for the curses library for both *nix systems and Windows, the interactive “vision” has become a reality.

What’s next for datascroller?

The Python package datascroller, currently for use with Pandas dataframes, will become the tool “datascroller” for general purpose terminal data scrolling. Imagine interactive terminal scrolling of any csv, text, or even JSON file that can be initiated from outside of Python. My past colleague John Merfeld, who makes extensive use of low vision accessibility tools, is on the project and will help consult as to whether certain color schemes (curses offers those) help make the terminal output easier to see, thus giving datascroller an accessibility angle.

Even with TerminalR, I could get around an R data frame pretty fast, fast than any GUI viewer. It has column and row searching functionality from the keyboard, and a lot of movement options. All these options and more are coming to datascroller soon, in full interactive fashion.

I have big plans for this tool.


Using the WordPress Template Hierarchy to Improve a Resume Plugin

Every Sole Proprietor’s website needs a resume, and given the expansiveness of the WordPress plugin ecosystem, I expected to see a half dozen to choose from. In reality, there’s only one with any amount of attention, Resume Builder from Boxy Studio.

I’m still working on the resume content, but the auto-formatted layout is not bad. The “star ratings” for skills are maybe a little corny but fun.

The plugin adds a “Resumes” section to your admin console screen, and when you add a new resume you’re treated to a very structured layout.

Don’t expect to drag and drop sections like they’re blocks in a WordPress editor. The plug in works, but doesn’t have that level of polish.

A resume that looks like a blog entry

The problem I encountered was that the resume looked like this:

Ridiculous! One copy of my mug per page is plenty. You can see from top left corner of the image that there’s a bit of WordPress debugging going on (live on the website, of course). I just couldn’t figure out why it was displaying like a blog article.

Well, if it looks like a duck… OK, when I’m editing my resume, the URL ends with post.php?post=75&action=edit. So resumes are just posts with a hard coded ID of 75. This explains why the shortcodes have 75 in them (e.g. rb-resume id="75" section="intro"). That left me with a dilemma, because I want at least the date on the blog articles (not sure about the mug), but I don’t want it on the resume.

Working harder, I echoed the post type in single.php by adding the following after the header:

<?php echo get_post_type(); ?>

When I clicked on the link for an regular blog article (which you have to do to trigger single.php. Going to the Blog link isn’t enough!), I’d see “post” echoed on the screen. But when I clicked on the resume, I saw “rb_resume.” Resume Builder is using a post type of “rb_resume.”

A Single.php for different post types

To solve the problem, I copied my single.php file into a new file called single-rb_resume.php. Since single.php calls another template part, I had to copy that file as well. And then I started hacking, and more in the machete sense than the computer programming one. Finally I was able to pull away the parts that made an individual blog post look like a blog post, and I arrived at the following:

Of course, instead of actually finishing the resume, I left satisfied with solving the templating problem; it’s still a work in progress.

The template hierarchy is a pretty scary part of WordPress. I can’t even begin to say I understand it, but I know that it’s there, and this situation has proven that exploiting the hierarchy can be a very powerful technique.


Getting a business bank account for a sole prop with a DBA

Category : The Journey

After applying for a DBA, I was confused whether or not I had actually received one, given the WI Register of Deeds sent back my filled out Registration of Firm Names form with a computer generated stamp on the top right corner. Surely that was just a receipt of payment, and something called a “Certificate of Assumed Name” was coming in the mail, right?

At least, that’s the idea I got based on interactions with “Bank A” (no reason to name names), my bank of over 10 years. Banker 1 from Bank A told me that I could only open a sole proprietor in my own name. I told him that I wanted a different name on the account, “Ogorek Data Sciences,” and that I was pursuing a Doing Business As (DBA) in order to do that. He seemed unsure but told me he could help me when “the paperwork” arrived. When the Registration of Firm Names form came back stamped, I sent Banker 1 a picture of the form. He responded that he absolutely needed the Certificate of Assumed Name to open the account.

So I waited, and watched the mailbox, and waited…

After a month of waiting, I went down to the Register of Deeds in person and asked what was going on. “We don’t give out certificates!” the woman behind (what I’m pretty sure was bulletproof) glass told me. “This is all you should need to open a bank account” and she gave be a printed Wisconsin.gov web page to prove it. Not convinced that Banker 1 knew the laws of the land, I made an appointment with Banker 2 (still of Bank A), armed with newfound confidence in my form and a printed out government web page with the exact instructions that I had followed. These instructions were clear.

Unfortunately, Banker 2 didn’t know much more than Banker 1. Again, she wanted a Certificate of Assumed Name, and I told her that I had fulfilled the requirements for the DBA according to the Wisconsin Register of Deeds and showed her the Wisconsin.gov printout. She still didn’t buy it and sent a photocopy of my form off to business documents review, which was supposed to take 24-48 hours.

Some 60 hours later, without a verdict, I went to “Bank B.” To Bank B’s credit, in an hour I had a business bank account, but a few things still bothered me. First, this third banker (“Banker 3”!) claimed that my Registration of Firm Names form was completely unnecessary, or at least he had been opening up accounts without them. He also told me that he didn’t think I could use an EIN for a sole prop (which I got a few weeks ago), but then he looked it up and realized I could use the EIN. “Hey, I learned something new!” Glad I could help.

To be fair to the first two bankers from Bank A, they were actually right about the account having to be opened in my name, but what they didn’t tell me was that I could still deposit checks written to the DBA name and the DBA name could appear on that account’s checks. If they did know this, then I suppose some blame lies with me for insisting on the account being opened in the DBA name. But come on, why else would I be so adamant about an account name?

And, to be fair to Banker 3, it turns out that the printed out Wisconsin.gov web page – that the Wisconsin Register of Deeds gave me in person – was from March of 2013. Maybe registering the Sole Prop is no longer required to open a bank account. And at least Banker 3 knew enough to convince me that I did indeed need to open the account in my own name, but with a DBA name attached to the account. (Seeing him type “Ogorek Data Sciences” after the “DBA” field in the application form, which made me feel better.)

It doesn’t seem like it should have been so hard. But, I’ve got a business bank account set up with a DBA name (“Ogorek Data Sciences”) and using an EIN. Mission accomplished.


Getting Health Insurance as a Freelancer

Category : The Journey

  • The Individual Mandate is no longer in effect as of 2019
  • Clicking the sponsored health care links on Google is bad
  • Individual health insurance that protects your wealth is inexpensive

Disclaimer: This author claims no special experience or knowledge of health insurance, other than the experience of losing corporate provided health insurance and having to buy new insurance (and screwing up a little along the way).

When you talk to someone who likes the idea of leaving their corporate job, two times out of three the person will use health insurance as the reason they can’t do it. When I did it this year, I’ll admit that I was worried about it myself, and back in 2015 when I took some time off, I paid around $300 a month for Affordable Care Act (ACA) qualifying health insurance that basically did nothing for me beyond wealth protection in the case of a really big bill. And it also let me avoid the individual mandate penalty on 2015 taxes.

Fast forward to June 1st, 2019, the first date I would lose corporate coverage, I was prepared to eat the $300 (or more) per month once again. Since I was losing coverage, I went on www.HealthCare.gov to try to apply during the enrollment off season. First, I wasted a lot of time by answering questions that could lead to discounts (they won’t for any remotely comfortable income, even if earned earlier in the year). Then, over a period of five days, the site was down and couldn’t even process my application. I thought I might have to eat one month of COBRA insurance for over $600 from my last job. Then I realized that the individual mandate is gone, at least at the federal level.

This is due to the Tax Cuts and Jobs act that was passed in December 2017 and eliminated the individual mandate penalty, effective January 1, 2019. It’d be a good idea to search for any law particular to your state, but nothing stood out as too powerful to me. It’s gone.

I have nothing to say about policy, or whether this is good or bad for the country, but I won’t buy expensive health insurance as a freelancer right after losing most of my income, especially when the site to do it isn’t even working.

After realizing I could just buy insurance from anyone, I got in a hurry and searched Google for “buying health insurance online.” Thinking that the top site was some kind of Kayak-like engine for comparison shopping health insurance, it clicked it. Do not do this. I will not write down the link out of fear that I will help these sites’ rankings, but I will show you a picture of what not to click:

The first website actually looked quite clean and professional, and made it easy for me to answer the few, very reasonable questions. I clicked submit.

Within, and I’m not kidding, 5 seconds, my phone started to ring. That’s when I knew I screwed up. Over the next three weeks, I got hundreds of phone calls and dozens of voicemails and emails. Telling them to stop doesn’t work:

It was pretty bad, but in the end it was only digital communications and not impossible to ignore (if you physically turned a few things off). Still, don’t do it.

In the end, I went to a website of a name I’ve seen before, United Health One, who’s offering non-ACA compliant “short-term” insurance. At first I didn’t understand what that meant, but when you’re buying the insurance you can specify different term lengths, and that seems to be about it. These policies are not ACA-compliant because they do almost nothing for you unless you’re out a lot of money (explaining the term “junk insurance”). But that’s okay for me.

What I really want is wealth protection in case something really bad happens and I owe a hospital $700,000, and thus I went for the highest maximum benefit of $2,000,000. The deductible situation is a little complicated because there are two of them. Under the first one the company will not pay anything, and in the sweet middle ground between the first deductible and second, they will pay some. I wasn’t that interested in that middle spot so I minimized the benefits there.

The UnitedHealthOne website (www.uhone.com) almost felt like one of those old school restaurants that offered so many million menu items by sheer combinatorics of the options, which I didn’t like so much. And it annoyed me a little bit that the company name changed from UnitedHealthOne to “Golden Rule” when it was time to buy the policy. But it was relatively easy to make the final purchase, I felt I got what I needed, and it was under $100 a month.

I’ll repeat the disclaimer: I have no idea what I’m doing. Maybe I do rack up $700,000 of medical expenses and then $400,000 of those expenses are denied by “Golden Rule.” Who knows? But I do know one thing: if you don’t click on one of those links I showed you, you can buy health insurance at a very reasonable rate without getting attacked by vultures. Sometimes, that’s all you can ask for.


Ogorek Data Sciences has an EIN

Category : The Journey

Most of the times, things are harder than they look, especially when those things involve the IRS. But I am pleasantly surprised and happy to say, getting an Employer Identification Number from www.irs.gov was amazing easy. IRS.gov has a nice page linking you to the online form, and after filling out the questions, you get the EIN in an instant. You can use the number immediately for most purposes (e.g., opening a bank account). For certain tax purposes, you’ll have to wait about two weeks. But not bad.

Why get an EIN as a sole proprietor?

Let’s be honest, I use the name “Ogorek Data Sciences” in blog post titles mostly in tongue-in-cheek fashion. This EIN is, for the foreseeable future, the Employer Identification Number of Ben Ogorek, the employer of himself. Furthermore, this sole proprietor has a Social Security Number that would work just fine for tax purposes. So why bother with it?

On the other hand, if I’m getting the boat-loads of 1099-MISC forms that I’m expecting in the near future, that’s a lot of Ben Ogorek-SSNs floating around on paper documents in the world. I’d rather replace those with the EIN. And, although I’m not in need of the “corporate veil” necessary for an LLC, it’s nice to practice treating the business as a separate entity with its own tax Id.

Cross it off the list!


WordPress theme changed to Enigma

Category : The Journey

I’ll never forget the response to the first “business website” I made. It was around 2009 and, fresh off an inspiration high from the 4-Hour Workweek, I cobbled together a comical-looking effort to siphon money out of the global economy. Despite not knowing anything about web technologies, I decided to build the website with VB.net and the free Microsoft development tools. In retrospect, I’m surprised I even got the site to work.

While WordPress surely existed back then, I didn’t know about it. Now, during my second at attempt at my own business, not only is the plan to actually create value, but to buy into trusted framework. I’ve been really impressed with how easy it is to get WordPress up and running, how cheaply it is to host, and how much it can look like a modern professionally-designed custom Website.

This morning, https://www.ogorekdatasciences.com looked like this:

After switching to the Enigma theme and doing just a bit of customization, the site looks like this:

I’m no marketer; some of the one-liners are a little goofy. But it’s a step in the right direction!


Applying for a DBA in Wisconsin

Category : The Journey

One of the choices I had to make as a data science freelancer was whether or not to register as a single-member LLC or proceed as a sole proprietor. A lot of people told me I should get the LLC for tax reasons, for example, to pay yourself a smaller income and take the remainder of the profits as a corporate distribution with a lower tax rates. However, all my research brought me to the conclusion that the type of LLC I’d be applying for, the single-member LLC, is a “pass-though entity,” and all of the income would pass through to my own personal income anyway.

Now the limited liability feature of the LLC is real, provided I’d be able to maintain a “corporate veil,” which I wasn’t sure I was ready to do at this early stage. In July, I’m thinking about forwarding my personal mail to my business address, for instance. So much for separation.

Since I still wanted the experience of creating a business that was not just my given name, I decided to go the DBA route. The DBA acronym stands for “Doing Business As” and allows a sole proprietor like me to create a legal business sounding name without the fees and hassle of an LLC. The DBA is necessary for opening bank accounts in that name as well (apparently you could get in trouble for using a business name that was not registered).

Applying for the DBA wasn’t hard, but like most things government related, you just have to know what to do. LegalZoom will charge you one hundred and change to take the guess work out of it, but if you just go to your state’s process, the instructions are usually pretty simple. In Wisconsin, you file a “Registration of Firm Name” with the county’s Register of Deeds. The instructions are pretty simple.

A week and a half ago, I sent in my notarized application requesting “Ogorek Data Sciences” as my sole proprietorship’s official along with the $30 dollar fee. The name’s not Don Draper grade, but I struggle with names and I asked myself, “do you want to waste time thinking of an awesome name, or do you want to get started?” And while I’ve always thought my last name came across as weird to people, a friend who makes fun of everything didn’t really laugh at the name, and another one commented that it’s pretty easy to say once you know how to say it. It’s pretty close to the sound of the name of the Oreck XL vacuum cleaner. Done.

About five days later, I got my same application back from the Register of Deeds but with the recording area stamped, showing that my payment was accepted. At first I asked, “is this it?” After some reading I found it can take a little longer than a couple days, and I take the returned application as a confirmation that my money is good at the Register of Deeds and my application has been sent to the next level. Godspeed.

Update: There was no “next level.” That was it. The Register of Deeds just recorded the firm name (I believe without any searching or anything.) This caused me some confusion when trying to use the form to get a business bank account with a DBA.