Diving for Data: a picture of APIs, ORM frameworks and servers (for visual people)
Programming involves the assimilation of abstract concepts. For me, the learning process reaches an apex of frustration when I try to imagine how these abstract concepts interact with one another. Each new definition seems like a solid shape until I add it to the sea of other abstract definitions in my mind, at which point it dissipates into uselessness.
I’ve realized that if I am going to survive in these waters I will need a visual picture of what is going on behind all those classes and instance variables. I will try to accomplish this using a (hopefully) interesting visual metaphor.
Diving for Data
Imagine a wealthy CEO, Mr. Matsuda, who has heard of a sunken ship off the coast of Japan, and wants to retrieve its treasure. He hires a team of divers, and an archeological expert, and commissions a vessel, The Ruby Python. He sends them out to the place where the sunken ship is known to be, and tells them to return to him with a report of what they find, plus a few samples.
The archeology expert, Ms. Tanaka, takes her team of divers to the coast and they begin their operation. Ms. Tanaka remains on board The Ruby Python and sends her divers down to take a look at the sunken ship, while she watches their progress on a monitor, using an undersea camera that sends pictures back to her on the boat.
The divers find the submerged ship, full of artworks and archeological treasures. There are sculptures, a large store of jewelry, a number of priceless Samurai swords, plus other valuable items. Ms. Tanaka calls the divers back and together they create a series of crates for storing the items they found. The divers then retrieve the treasure from the sunken ship, bring it back up to The Ruby Python and place it in the crates. Finally, they clean and wrap the samples, and bring them back to Mr. Matsuda.
What does this have to do with programming?
Everything in this story can be a visual image for the way data is retrieved from a server, stored in an application like Ruby on Rails, and finally made visible to the user.
Let’s start with SQL. SQL stands for Structured Query Language, and is the language used to query and update relational databases. In the story, think of SQL as the question, the impulse to explore, the actions taken by the CEO and the divers. “Is there buried treasure?” they ask. “Where is the treasure?” When they find it, there is a command: “Bring the treasure back.”
How is this done? The first necessary aspect of treasure-retrieval is the divers themselves. These divers can be compared to APIs. API stands for Application Programming Interface. The divers function as the interface between the sunken ship, which is like the server, and the archeological vessel, The Ruby Python, which is like the client.
Let’s assume that, like APIs, the divers can speak to each other. Perhaps they are using Morse code to communicate basic information to each other (and to Ms. Tanaka on the ship). This Morse code is like JSON, a way to communicate pure data — from server to server or from server to client — via the divers (the APIs). These divers can make dives into any number of sunken vessels during their expedition. They can communicate with other dive teams. In the same way, APIs can bring data up from different databases and make it available to a number of clients.
Now the divers have information about what’s in the sunken vessel and they can bring that information back. Similarly, using APIs, we can know what’s in our database, just like the divers know what’s in the ship. But it’s not enough just to have the information. JSON is just a bunch of data. But Mr. Matsuda wants a sample. Just like the user of a website wants to see and interact with the server data in a way that makes sense to him / her. So, we need some way to carry it back to them. This is where our ORM (Object-Relational Mapping) framework comes in.
Think of The Ruby Python as the client. It has three parts.
- First, there is Ms. Tanaka, who interacts with Mr. Matsuda. She serves two functions. One one hand, she is like the code in the view, because part of her function as archeological expert is to interpret the treasure to Mr. Matsuda, to tell him its value, its age, from which region of Japan it originated, and so on. On the other hand, her function is also to bring the treasure to him, to allow him to touch and experience it. In this way, she is the view, and also the browser. (We can also think of her undersea-camera monitor like AJAX, which is a way for the browser to see the contents of a server in real time).
- The Ruby Python itself is also part of the client. It is the thing that carries the divers and makes their mission possible, and also carries the treasure in its hold back to the mainland. It is like Ruby on Rails, or your framework of choice, carrying the APIs, SQL commands, and objects inside its code.
- The third part of the client is the storage capacity on board The Ruby Python. The ship is carrying a bunch of empty crates. These empty crates are labeled “sculpture” or “jewelry” or “swords”. These crates are like our ORM framework. We can send a diver down to the ship and he can bring up some swords and place them in the Swords crate. This is a basic picture of the way an API can interface with a server (via a SQL / REST request) and bring back objects (and their related information) and place them in a pre-constructed Class inside your Rails (or other) framework. The Swords Class will have instances of swords, the Jewelry Class will have instances of jewelry, and so on.
Finally, once Ms. Tanaka has seen these objects, she can perform actions on them. She can organize them. She can wrap them. And she can deliver them to Mr. Matsuda, in his fancy office on the 47th floor of his corporate offices in Tokyo.
In the same way, in the view we can manipulate objects, put them in order, wrap them in pretty CSS and deliver them to the user by way of the browser.
Disclaimer: I am a new programmer and I understand my metaphor might be flawed / imprecise. I wrote this post as a way to better grasp these concepts! Experienced input is welcome. If you have any corrections or can find a way to add a Kraken to my story, please do so in the comments. Thank you!