My CLI Data Gem Project

Posted by 81Jeremiah on May 17, 2018

Wow, I did it, I created my first gem! This first project was an incredible experience for me. Albeit, it was a little of a rocky at times, but I was able to make to create something that I am proud of. The biggest tip for other students starting this journey is to watch the daily deal video walkthrough. Avi discusses a process of going back and forth from broken to fixed that really helped me to build each part of my gem piece by piece that made the task much easier to complete. To start my journey, I had to take the leap to move away from the comfort of the learn IDE. I know some students may have already started to work out of their terminal, but the IDE is all I knew. Programming out of my terminal was a whole new world and the concept of it was quite overwhelming. I started by downloading sublime and then built a new gem with bundler to create the skeleton of me gem. Like magic all of the basic folders for my gem appeared. Next I had to give this new folder permissions to run on my system. Thanks to the learn video, this was pretty easy to get going. My terminal was allowing me to run my gem and I was ready to start writing some code.

As mentioned above the system of “broken to fixed” worked very well for me. I started with my executable file and created a quick a quick CLI class. After messing around with a few “require relatives” I was in business and my gem was outputting some hard-coded text. My gem scrapes History channel’s “Today in History” site. So, I first built a very basic class for events that had attributes for year, title, category and main story. My plan was to build a scraper that when it scraped the site would create the objects in my Event class and store all of my individual objects(events) in an array. My plan was solid but my ability to scrape was not. I spent hours on the scrape. When it all came down to it my issue all happened because of an ‘s’. When I scraped the site, I didn’t go in deep enough to my css attribute to irritate through my scrape properly. I selected “day-articles” instead of “day-article”. This little error left with me hours of running every combination through pry to get individual strings, but I just kept getting giant chunks of text. I tried turning those into arrays and then tried to irritate through those to build my objects, but that turned out to be an even bigger disaster. Finally, after another hour I decided ask for help. By the time my 1:1 started I had become a crazy person with about 20 different windows open and commented out code everywhere, trying to find where my error was. My very calm technical coach nudged me in the right direction to look at the class selector I choose and try again. Once I did, everything worked! I learned a lot through my brief dive into insanity, 1st I probably should have asked for help earlier, but also to recheck my work before going off the deep end. I had tried that css selector before, but I didn’t have the individual selector for the div correct. It was a simple mistake that costs me hours. It’s always nice to have a second pair of eyes to look at your code!

Once my scraper worked I was building objects without a problem. From there I only encountered two major problems. The first one was pretty simple to fix. I had this odd issue with the lead story duplicating itself in my list. To fix this I just told the method to not add any events that already had an attribute category of “Lead Story” in the array. The other issue took a little bit of strategy and slightly deeper thinking. For this problem only the Lead Story had all the text for the story on the page I was scraping. This meant I had to somehow tell the method to open up each events individual link to scrape each events page. Sounds simple now ad after a little bit of research to come to that understanding that is exactly how I solved the problem. I created an if statement within the method that made my events, that would open up the individual event page and scrape the content I needed. To do this I created a link attribute and interpolated that link so it added to my main link to open up the new page. Once again, this only applied to the additional stories, so I made sure if the category was “Lead Story” the method wouldn’t preform this task on that particular object.

After some more back and forth from “broken to fixed” in my CLI class I was satisfied with my end results and submitted my link, what a feeling of accomplishment! Now that this project is finished I am left with a deeper understanding of nokogiri, objects, using my computer’s terminal and most importantly a better understanding of how I solve problems and a better understanding of the process to build more programs in the future.

Please feel free to check out my gem here- https://github.com/81Jeremiah/this_day_in_history-cli-app