Project: Event manager
Learn how to build an event management app that pulls information from an external API.
Last updated
Learn how to build an event management app that pulls information from an external API.
Last updated
This tutorial has been adapted from The Turing School's and Jump Start Lab's and updated to use Google's Civic Information API.
After completing this tutorial, you will be able to:
Manipulate input and output
Read content from a (Comma Separated Value) file
Transform it into a standardized format
Utilize the data to contact a remote service
Populate a template with user data
Manipulate
Access through
the
Use (Embedded Ruby) for templating
Imagine that a friend of yours runs a non-profit organization around political activism. A number of people have registered for an upcoming event. She has asked for your help in engaging these future attendees. For the first task, she wants you to find the government representatives for each attendee based on their zip code.
Create a project directory folder named event_manager wherever you want to store your project. In the project directory, create another folder named lib and inside that folder create a plain text file named event_manager.rb. Using your command-line interface (CLI), you can enter the following commands:
Creating and placing your event_manager.rb file in lib directory is entirely optional; however, it adheres to a common convention within most ruby applications. The filepaths we use in this tutorial will assume that we have put our event_manager.rb file within the lib directory.
Ruby source file names should be written all in lower-case characters and instead of camel-casing multiple words together, they are instead separated by an underscore (often called snake_case).
Open lib/event_manager.rb in your text editor and add the line:
Validate that ruby is installed correctly and you have created the file correctly by running it from the root of your event_manager directory:
If ruby is not installed and available on your environment path then you will be presented with the following message:
If the file was not created then you will be presented with the following error message:
If this happens, make sure the correct directory exists and try creating the file again.
For this project we are going to use the following sample data:
After the file is downloaded, you should see something like:
The first few rows of the CSV file you downloaded look like this:
In this example, it does not matter whether you use single quotes or double quotes. There are differences, but they are essentially the same when representing a string of characters, such as this initial greeting or the name of the file.
We are assuming the file is present here. However, it is a good practice to confirm that a file exists. File has the ability to check if a file exists at the specified filepath on the filesystem through File.exist? "event_attendees.csv".
First, we read the entire contents of the file as an array of lines. Second, we iterate over the array of lines and output the contents of each line.
Instead of outputting the entire contents of each line we want to show only the first name. That requires us to look at the current contents of our Event Attendees file.
The first row contains header information. This row provides descriptional text for each column of data. It tells us the data columns are laid out as follows from left-to-right:
ID - the empty column represents a unique identifier or row number of all
the subsequent rows
RegDate - the date the user registered for the event
first_Name - their first name
last_Name - their last name
Email_Address - their email address
HomePhone - their home phone number
Street - their street address
City - their city
State - their state
Zipcode - their zipcode
The lack of consistent formatting of these headers is not ideal when choosing to model your own data. These column names are our extreme example of a poorly formed external service. Great applications are often built on the backs of such services.
We are interested in the first_Name column. At the moment, we have a string of text that represents the entire row. We need to convert the string into an array of columns. The separation of the columns can be identified by the comma "," separator. We want to split the string into pieces wherever we see a comma.
Within our array of columns we want to access our 'first_Name'. This would be the third column or third element at the array's second index columns[2]. Remember, arrays start counting at 0 instead of 1, so columns[0] is how we would access the array's first element, and columns[2] will give us the third.
The header row was a great help to us in understanding the contents of the CSV file. However, the row itself does not represent an actual attendee. To ensure that we only output attendees, we could remove the header row from the file, but that would make it difficult if we later returned to the file and tried to understand the columns of data.
Another option is to ignore the first row when we display the names. Currently we handle all the rows exactly the same, which makes it difficult to understand which one is the header row.
One way to solve this problem would be to skip the line when it exactly matches our current header row.
When this program is run, the next if line checks every line to see if it matches the provided string. If so, it skips that line from the rest of the loop
A problem with this solution is that the content of our header row could change in the future. Additional columns could be added or the existing columns updated.
A second way to solve this problem is for us to track the index of the current line.
This solves the problem if the header row were to change in the future. It assumes that the header row is the first row in the file.
Either of these solutions would be OK given our current attendees file. Problems may arise if we are given a new CSV file that is generated or manipulated by another source. This is because the CSV parser that we have started to create does not take into account a number of other features supported by the CSV file format.
Two important ones:
CSV files often contain comments which are lines that start with a pound (#) character
A column is unable to support a value which contains a comma (,) character
Our goal is to get in contact with our event attendees. It is not to define a CSV parser. This is often a hard concept to let go of when initially solving a problem with programming. An important rule to abide by while building software is:
Look for a Solution before Building a Solution
Ruby actually provides a CSV parser that we will use instead throughout the remainder of this exercise.
It is likely the case that if you want to solve a problem, someone has already done it in some capacity. They may have even been kind enough to share their solution or the tools that they created. This is the kind of goodwill that pervades the Open Source community and Ruby ecosystem.
Ruby's core language comes with a wealth of great classes. Not all of them are loaded every single time ruby code is executed. This ensures unneeded functionality is not loaded unless required, preventing ruby from having slower start up times.
First we need to tell Ruby that we want it to load the CSV library. This is done through the require method which accepts a parameter of the functionality to load.
Instead of read or readlines we use CSV's open method to load our file. The library also supports the concept of headers and so we provide some additional parameters which state this file has headers.
There are pros and cons to using an external library. One of the pros is that this library makes it easy for us to express that our file has headers. One of the cons is that we have to learn how the library is implemented.
CSV files with headers have an additional option which allows you to access the column values by their headers. Our CSV file defines several different formats for the column names. The CSV library provides an additional option which allows us to convert the header names to symbols.
Converting the headers to symbols will make our column names more uniform and easier to remember. The header "first_Name" will be converted to :first_name and "HomePhone" will be converted to :homephone.
Accessing the zipcode is very easy using the new header name, :zipcode.
We now are able to output the name of the individual and their zipcode.
Now that we are able to visualize both pieces of data we realize that we have a problem....
The zip codes in our small sample show us:
Most zip codes are correctly expressed as a five-digit number
Some zip codes are represented with fewer than five digits
Some zip codes are missing
Before we are able to figure out our attendees' representatives, we need to solve the second issue and the third issue.
Some zip codes are represented with fewer than five digits
So in the case of zip codes of fewer than five digits, we will assume that we can pad missing zeros to the front.
Some zip codes are missing
Some of our attendees are missing a zip code. It is likely that they forgot to enter the data when they filled out the form. The zip code data was not likely marked as mandatory and so our future attendees were not presented with an error message.
We could try to figure out the zip code based on the rest of the address provided. But we could be wrong with our guess, so instead we will use a default, bad zip code of "00000".
Before we start to explore a solution with Ruby code it is often helpful to express what we are hoping to accomplish in English words.
If the zip code is exactly five digits, assume that it is ok
In the case when the zip code is five digits in length we have it easy. We simply want to do nothing.
If the zip code is more than five digits, truncate it to the first five digits
While zip codes can be expressed with additional resolution (more digits after a dash), we are only interested in the first five digits.
If the zip code is less than five digits, add zeros to the front until it
becomes five digits
There are many possible ways that we can solve this issue. These are a few paths:
Use a while or until loop to prepend zeros until the length is five
Calculate the length of the current zip code and add missing zeros to the front
Add five zeros to the front of the current zip code and then trim the last five digits
The following solution employs:
When we run our application, we see the first few lines output correctly and then the application terminates.
What is the error message "undefined method 'length' for nil:NilClass (NoMethodError)" saying?
Reviewing our CSV data, we notice that the next row specifies no value. An empty field translates into a nil instead of an empty string. This is a choice made by the CSV library maintainers. So we now need to handle this situation.
Our solution above does not handle the case when the zip code has not been specified. CSV returns a nil value when no value has been specified in the column. All objects in Ruby respond to #nil?. All objects will return false except for a nil.
We can update our implementation to handle this new case by simply adding a check for nil?.
It is important for us to take a look at our implementation. During this examination we should ask ourselves:
Does the code clearly express what it is trying to accomplish?
The implementation does a decent job at expressing what it accomplishes. The biggest problem is that it is expressing this near so many other concepts. To make this implementation clearer we should move this logic into its own method named clean_zipcode.
This may feel like a very small, inconsequential change, but small changes like these help make your code cleaner and your intent clearer.
With our clean zip code logic tucked away in our clean_zipcode method, we can examine it further to see if we can make it even more succinct.
Coercion over Questions
Combining all of these steps together, we can write a more succinct clean_zipcode method:
The Civic Information API allows registered individuals (registration is free) to obtain some information about the representatives for each level of government for an address.
For any U.S. residential address, you can look up who represents that address at each elected level of government. During supported elections, you can also look up polling places, early vote location, candidate data, and other election official information.
Take a close look at this sample URL for accessing the Civic Information API:
Here's how it breaks down:
https:// : Use the Secure HTTP protocol
www.googleapis.com/civicinfo/v2/ : The API server address on the internet
representatives : The method called on that server
? : Parameters to the method
& : The parameter separator
address=80203 : The zipcode we want to lookup
levels=country : The level of government we want to select
roles=legislatorUpperBody : Return the representatives from the Senate
roles=legislatorLowerBody : Returns the representatives from the House
key=AIzaSyClRzDqDh5MsXwnCWi0kOiiBivP6JsSyBw : A registered API Key to authenticate our requests
When we're accessing the representatives method of their API, we're sending in a key which is the string that identifies JumpstartLab as the accessor of the API, then we're selecting the data we want returned to us using the address, levels, and roles criteria. Try modifying the address with your own zipcode and load the page.
Let's look for a solution before we attempt to build a solution.
Ruby comes packaged with the gem command. This tool allows you to download libraries by simply knowing the name of the library you want to install.
Reading through the documentation on how to set up and use the google-api-client gem, we find that we need to perform the following steps:
Set the API Key
Send the query with the given criteria
Parse the response for the names of your legislators.
Exploration of data is easy using irb:
Whoa. That's a lot of information. Buried in there are the names of our legislators. We can access them by calling the .officials method on the response. Now that we know how to access the information we want, we can focus our attention back on our program.
Running our application, we find an error.
The legislators that we are displaying is an array. In turn, the array is sending the to_s message to each of the objects within the array, each legislator. The output that we are seeing is the raw legislator object.
We really want to capture the first name and last name of each legislator.
Instead of outputting each raw legislator we want to print only their first name and last name. We will need to complete the following steps:
For each zip code, iterate over the array of legislators.
For each legislator, we want to find the representative's name.
Add the name to a new collection of names.
We can further simplify this into its final form:
If we were to replace legislators with legislator_names in our output, we would be presented with a slightly better output.
The problem now is that when using string interpolation, Ruby is converting our new array of legislator names into a string, but Ruby does not know exactly how you want to display the contents.
Running our application this time should give us a much more pleasant looking output:
Similar to before, with this step complete, we want to look at our implementation and ask ourselves:
Does the code clearly express what it is trying to accomplish?
This code is fairly clear in its understanding. It is simply expressing its intent near so many other things. It is also expressing itself differently from how zip codes are handled. This dissimilarity breeds confusion when returning to the code.
We want to extract our legislator names into a new method named legislators_by_zipcode, which accepts a single zip code as a parameter and returns a comma-separated string of legislator names.
An additional benefit of this implementation is that it encapsulates how we actually retrieve the names of the legislators. This will be of benefit later if we decide on an alternative to the google-api gem or want to introduce a level of caching to prevent look ups for similar zip codes.
We have our attendees and their respective representatives. We can now generate a personalized call to action.
For each attendee we want to include a customized letter that thanks them for attending the conference and provides a list of their representatives. Something that looks like:
We could define this template as a large string within our current application.
Ruby has quite a few ways that we can define strings. This format %{ String Contents } is one choice when defining a string that spans multiple lines.
However, placing this large blob of text, this template, within our application will make it much more difficult to understand the template and the application code and thus make it more difficult to change the template and the application code.
Instead of including the template within our application, we will instead load the template using the same File tools we used at the beginning of the exercise.
Create a file named 'form_letter.html' in the root of your project directory.
Copy the html template defined above into that file and save it.
Within our application we will load our template:
It is important to define the form_letter.html file in the root of project directory and not in the lib directory. This is because when the application runs it assumes the place that you started the application is where all file references will be located. Later, we move the file to a new location and are more explicit on defining the location of the template.
gsub and gsub!For each of our attendees we want to replace the FIRST_NAME and LEGISLATORS with their respective values.
We need to find all instances of FIRST_NAME and replace them with the individual's first name.
We need to find all instances of LEGISLATORS and replace them with the individual's representatives.
These two methods are almost identical save for one important difference. The method gsub returns a new copy of the original string with the values replaced. Whereas gsub! will replace the values in the original string.
We have our template letter which we want to use for every attendee. It is important that we create a new copy of this letter for each attendee. If we change the original template, they'd all have the same name! By making a copy and then changing the copy, we're sure everyone's name is unique.
Methods like gsub and gsub! can often be confusing and when to use one over the other may not be immediately clear. The above template manipulation could have been written with just gsub:
It is a treacherous road we start to walk, defining our own templating language. Our current system has some flaws:
Using FIRST_NAME and LEGISLATORS to find and replace might cause us problems if later somehow this text appears in any of our templates.
Though not likely, imagine if a person's name contained the word 'LEGISLATORS'. When we perform the second replacement operation, that part of the person's name would also be replaced. This is unlikely in our simple template, but as our template grows, we may invite such disasters.
We cannot represent multiple items very easily if they are surrounded by HTML.
Currently we copied our legislators string into a single table column. We would have a hard time inserting our legislators as individual rows in the table without having to build parts of the HTML table ourself. This could spell disaster later if we decide to change the template to no longer use a table.
So again, instead of building our own custom solution any further, we are going to seek a solution.
ERB provides an easy to use but powerful templating system for Ruby. Using ERB, actual Ruby code can be added to any plain text document for the purposes of generating document information details and/or flow control.
Defining an ERB template is extremely similar to our existing template. The difference is that we use escape sequence tags which allow us to insert any variables, methods or ruby code we want to execute.
ERB defines several different escape sequence tags that we can use. The most common are:
We can define our ERB escape tags within any string. The ruby defined within the contents of the ERB tags will not be evaluated until we ask the template to give us the results.
The code above loads the ERB library, then creates a new ERB template with the question string. The question string contains ERB tags that will show the results of the variable meaning_of_life. We send the result message to the template with binding.
What is binding?
Having to explicitly specify a binding when we ask for the results of the template gives us the flexibility to ask for the results of a template given a different binding.
To use ERB we need to update our current template form_letter.html.
Save a new template as form_letter.erb
The convention is to save ERB template files with the extension erb. This is not a requirement. It is a benefit to yourself and other users when they return to the application.
Update our existing keywords with the ERB escape sequences
The use of the ERB tags to display the attendee's name is familiar from our previous example. The second use, when we display the legislators, is different. We are using the ERB tag that does not output the results <% %> to check if the legislators variable is an Array.
If it is an array, we output the name and website url of each legislator. This is a departure from what we originally implemented. Before we had to build the names of all the representatives. We intend now to give the template direct access to the array of legislators. We will let the template ask and display what it wants from each legislator.
If legislators is not an array, it means that the legislators_by_zipcode method entered the rescue clause, which outputs a string. We simply want to display that string.
We now need to update our application to:
Require the ERB library
Create the ERB template from the contents of the template file
Simplify our legislators_by_zipcode to return the original array of legislators
Require the ERB library
First we need to tell Ruby that we want it to load the ERB library. This is done through the require method which accepts a parameter of the functionality to load.
Create the ERB template from the contents of the template file
Creating our template from our new template file requires us to load the file contents as a string and provide them as a parameter when creating the new ERB template.
Simplify our legislators_by_zipcode to return the original array of legislators
Outputting each form letter to the screen was useful for ensuring our output looked correct. It is time to save each form letter to a file.
Each file should be uniquely named. Fortunately, each of our attendees has a unique id—the first column, or row number.
Assign an ID for the attendee
Create an output folder
Save each form letter to a file based on the id of the attendee
Assign an ID for the attendee
The first column does not have a name like the other columns, so we fall back to using the index value.
Create an output folder
We make a directory named "output" if a directory named "output" does not already exist.
Save each form letter to a file based on the id of the attendee
Again, for the sake of writing clean and clear code, we want to move the operation of saving the form letter to its own method:
The method save_thank_you_letter requires the id of the attendee and the form letter output.
Similar to the zip codes, the phone numbers suffer from multiple formats and inconsistencies. If we wanted to allow individuals to sign up for mobile alerts with the phone numbers, we would need to make sure all of the numbers are valid and well-formed.
If the phone number is less than 10 digits, assume that it is a bad number
If the phone number is 10 digits, assume that it is good
If the phone number is 11 digits and the first number is 1, trim the 1 and use the first 10 digits
If the phone number is 11 digits and the first number is not 1, then it is a bad number
If the phone number is more than 11 digits, assume that it is a bad number
The boss is already thinking about the next conference: "Next year I want to make better use of our Google and Facebook advertising. Find out which hours of the day the most people registered, so we can run more ads during those hours." Interesting!
Using the registration date and time we want to find out what the peak registration hours are.
Explore the documentation to become familiar with the available methods, especially #strptime, #strftime, and #hour.
The big boss gets excited about the results from your hourly tabulations. It looks like there are some hours that are clearly more important than others. But now, tantalized, she wants to know "What days of the week did most people register?"
If this happens, see .
Download the csv file and save it in the root of the project directory, event_manager. Using your CLI, confirm that you are right directory and enter the following command:
A comma-separated values file stores tabular data (numbers and text) in plain-text form. The CSV format is readable by a large number of applications (e.g. Excel, Numbers, Calc). Its portability makes it a popular option when sharing large sets of tabular data from a database or spreadsheet applications.
is a core ruby class that allows you to perform a large number of operations on files on your filesystem. The most straightforward is File.read.
Reading and displaying the entire contents of the file showed us how to quickly access the data. Our goal is to display the first names of all the attendees. There are numerous methods that would allow us to manipulate this large string. Using File.readlines will save each line as a separate item in an array.
Ruby's allows you to convert a string of text into an array along a particular character. By default when you send the split message to the String without a parameter it will break the string apart along each space " " character. Therefore, we need to specify the comma "," character to separate the columns.
This is a such a common operation that Array defines .
In this iteration we are going to convert our current CSV parser to use Ruby's . We will then use this new parser to access our attendees' zip codes.
You can browse the many libraries available through the .
The way loads and parses data is very similar to what we previously defined.
If we looked at the , we would see that the majority of the shorter zip codes are from states in the north-eastern part of the United States. Many zip codes there start with 0. This data was likely stored in the database as an integer, and not as text, which caused the leading zeros to be removed.
Use to append zeros to the front of the string.
- returns the length of the string.
- to pad the string with zeros.
- to create sub-strings either through the slice method or the array-like notation []
A good rule when developing in Ruby is to favor coercing values into similar values so that they will behave the same. We have a special case to deal specifically with a nil value. It would be much easier if instead of checking for a nil value, we convert the nil into a string with .
Examining in irb, we can see that when we provide a string with a length greater than five, it performs no work. This means we can apply it in both cases, as it will have the same intended effect.
Lastly, examining in irb, we can see that for a number that is exactly five digits in length, it has no effect. This also means we can apply it in cases when the zip code is five or more digits, and it will have the same effect.
We now have our list of attendees with their valid zip codes (at least for most of them). Using their zip code and the , we are able to query for the representatives of a given area.
This document is formatted. If you copy and paste the data into a , you can see there is an officials key that has many legislator names. The response also includes a lot of other information. Cool!
If you receive a signet error when installing the Google API gem, it is due to modern Ruby updates requiring an updated version of signet that is not compatible with the API. To fix, please before installing the gem.
The gem comes equipped with some vague example documentation. The documentation is also available online with their .
What does this mean? It means that the Google API was unable to use an address we gave it. When we dig further, we see that right before this error the information from David with a zip code of 07306 is printed. Looking at the data we can now see that the attendee after David did not enter a zip code. Data missing like this is common, so we have to have a way of dealing with it. Luckily, Ruby makes that easy with its . We can add a begin and rescue clause to the API search to handle any errors.
To do this, we can use the function built into ruby. It works just like .each but returns a new array of the data we want to include.
We need to explicitly convert our array of legislator names to a string. This way we are sure it will output correctly. This could be tedious work except Array again comes to the rescue with the method.
allows the specification of a separator string. We want to create a comma-separated list of legislator names with legislator_names.join(", ")
Our template is a String of text which has two methods for replacing text: and .
We replace the first name in the template letter and return a new copy (Thanks ). We save the new letter to a personal version of the letter personal_letter. We then replace all the legislators with our legislators information in personal_letter (Thanks ).
Ruby defines a template language named .
The method returns a special object. This object is an instance of . An instance of binding knows all about the current state of variables and methods within the given scope. In this case, binding knows about the variable meaning_of_life.
allows us to open a file for reading and writing. The first parameter is the name of the file. The second parameter is a flag that states how we want to open the file. The w states we want to open the file for writing. If the file already exists it will be destroyed.
Afterwards we actually send the entire form letter content to the file object. The file object responds to the message puts. The method , from which the file object inherits, is similar to which we have been using up to this point.
Ruby has and classes that will be very useful for this task.
For a quick overview, check out this article.
Use to find out the day of the week.