Month: August 2010

online database of DS1 and DS3 special access rates

The DS1 and DS3 rates that the Ad Hoc Telecommunications Users filed publicly at the FCC are now accessible as an online, highly capable Needle domain (database).  Needle is a data system that makes it easy to look at the data in different ways and to sort and filter it, all from within a web browser.

The original filings (here, here, and here) provide the data as pdf pages displaying tables with highly complex row and column structures. A human can read and page through the data as if it were text.  That data format serves neither the reading capabilities of humans nor the data-processing capabilities of computers.

To make the Ad Hoc DS1/DS3 rate data more accessible,  I extracted it from the pdf files and re-organized it into one, regular, comma-separated-value (CSV) file with 3698 data rows.  I also put together some relevant data documentation. Analyzing the CSV file with a spreadsheet is possible but cumbersome. Since the CSV file has a simple tabular data form, it’s easy to analyze with a database program, if you have one. You would download the data, import it into the database program, and then set up and run a query that generated the data view that you seek.

Needle makes many different views of the data easily accessible to a web browser.  Within Needle, a dataset is a graph of data nodes, where each data node is a single piece of data of a particular type. The Needle Ad Hoc DS1/DS3 domain shows (on the left under “Every:”) a linked list of every node type in that dataset. If you click on any of these node types, a table will appear that has as its leftmost column a list of all the data nodes of the clicked type. So, for example, if you click on “bandwidth,” you will see the nodes DS1 and DS3 in the left column of the table.  The table also shows the number of attribute sets and the average circuit10 rate (a composite rate) across the DS1 and DS3 nodes, respectively.  You can look at the circuit10 rates by clicking on the circuit10 link (node type) on the left. The resulting table shows all the circuit10 rates, in descending order, in the left column.  Other columns of the table show other attributes associated with each circuit10 rate.

For any table that you see, you can filter, sort, and group the data.  For example, to limit the table of circuit10 rates to DS1 rates, left click on the “bandwidth” column heading, select “filter by this column” in the pop-up menu, type DS1 into the box next to “show”, and then click on “do” just to the right of that box. The table will then contract to show just the DS1 circuit10 rates.  A similar procedure produces filters for company, year, state, reg type, term, and zone.  If you want to see the elements of each of these data types, click on that type on the left. Options on the pop-up menu also provide for sorting and grouping.  Under “Index” on the top left, the “rates” and “rates subset” links show examples of tables made from grouping, filtering, and  sorting the cn (attribute set) nodes.  The “compare 2009 to 2006” and “compare 2009 to 2005” links under the index heading show tables that include circuit10 price ratios across the relevant years.  You can sort and filter these tables like any other table.

Any subset of data can be extracted easily from Needle.  At the bottom of each table are links “See this data as: Plain List · CSV · JSV · JSONa”.  Just click on CSV to download a CSV file of the data.  If the table has groups, you need to flatten the table (switch grouping to a regular data column) before exporting.  Needle also offers API functionality that allows Needle to serve as a data repository for high-powered statistical analysis packages such as R or S.

Needle can do much more than what it is doing for the Ad Hoc DS1/DS3 dataset.  Needle’s strengths include data acquisition, merging, and cleansing.  In addition, Needle’s graph-based data organization can easily handle complex data structures that create nightmares in traditional relational databases, which require tabular data forms.  Needle, for example, can easily handle variable-length lists of items.  None of these strengths are applied to present the Ad Hoc DS1/DS3 dataset. Needle here merely makes the Ad Hoc DS1/DS3 data much more easily accessible, especially compared to data published as pages of tables in a pdf document.

making art exhibits accessible

Jon Berge's Four/For Mona Lisa

The museum guard with a gun said, “Put your camera away. No photographs in the exhibit.”  That’s what I remember most about my first visit to the exhibit Revealing Culture, now at the Smithsonian Institute in Washington DC.  VSA, the International Organization on Arts and Disability, sponsored the exhibit.  The guard was just doing his job.  I respect the many, dedicated bureaucrats who diligently do their jobs.

While I was dismayed that the Corcoran Gallery forbade non-commercial photographs of Maya Lin’s re-formations of natural landscapes, forbidding photographs at Revealing Culture is worse.  VSA was founded “to provide arts and educational opportunities for people with disabilities and increase access to the arts for all.”  What about persons who are bedridden and can’t physically come to the exhibit?  What about those who don’t have money or time to travel to the exhibit?

Preventing art exhibit visitors from making non-commercial photographs signals an exhibit’s avariciousness for artifacts and contempt for art.  Constraints are a natural feature of the real world.  But reasonable accommodations are possible.  Allowing visitors to make and share photographs of an exhibit costs the organizers neither time nor money.  Sharing of non-commercial photographs through social networks can bring a sense of the exhibit to many persons who otherwise would have none.  Sharing enhances true art.  Art exhibits that prohibit non-commercial photographs seem to have an anti-social obsession with possession.

Change is possible.[*]  Earlier this week, strolling about the area, I again entered Revealing Culture. A museum guard was there.  Still unwillingly suspended in disbelief, I asked him if I could take photographs.  He said, “Yes, in here you can.”  Wonderful!  An art exhibit that allows sharing culture!

My favorite work in Revealing Culture is Jon Berge’s Four/For Mona Lisa. On the wall facing the exhibit entrance, it resembles a band of shields that riot police use in charging a crowd.  But in reality the shields are boards offering braille representations of inner-city children describing the Mona Lisa:

  • Mona Lisa is a lady in a picture.  Bright eyes, pretty smile.  She has no family.  She is outside in the sun.
  • The Mona Lisa probably took a long time to draw so I bet she was very bored – and didn’t want to stay any longer.  it only shows from the waist and up of her body.  the Mona Lisa has on eyebrows on her
  • She looked like she wanted to kill Leonardo for making her pose for a picture and she doesn’t dress like us.  Leonardo must be her husband.
  • Mona Lisa looks sad because she is alone.  She is poor

Touching Four/For Mona Lisa activates a digital recording of the children reading their descriptions.  Reality gives of itself to everyone who seeks it, to each in a way that relates to her or his capabilities.  Jon Berge’s Four/For Mona Lisa is art of reality.

feeling Emily Eifler's art

Revealing Culture is at the Smithsonian’s Ripley Center in Washington, DC, through August 29, 2010.

* * * * *

[*] A VSA publication, Arts Access Made Easy: Successful Strategies from the Award of Excellence in Arts Access, offers ten ways to achieve arts access.  It seems oblivious to the rapidly developing capabilities of digital networks.  Concern about access should include concern about being accessible through the Internet.  Moreover, Arts Access Made Easy is oriented toward organizational-bureaucratic actions.  Fostering grass-roots digital representations and peer-to-peer sharing across social networks can make a large contribution to improving access to a given body of work.

the video revolution

you are a television network

Anyone can now make and distribute video world-wide at zero incremental cost.  That’s a mind-boggling video communication revolution.

Google’s recent heroic effort to count the number of books that have been published around the world shows how scarce video has been.  Google estimates tomes — symbolically distinguished, printed and bound works — to number 146 million.  Google also estimates the number of video works in library catalogs to be about 2 million.[*]  Hence the number of video works equals less than 2% of the number of tomes that the world’s libraries hold.

Other public library statistics indicate popular interest in video.  U.S. public libraries’ video holdings amounted in 2008 to 5.4% of total items held. These item counts include duplicates of works within a library and across libraries.  The higher video item share compared to the video work share suggests that U.S. libraries include more duplicate video items than duplicate print items from the world population of tomes and videos.  Moreover, U.S. libraries’ video circulation accounts for about 30% of total item circulation in 2010. The higher circulation share compared to item share for videos indicates that videos are borrowed more frequently than other items.

Public library holdings represent institutionally authorized items.  Commercial video rentals, which are about five times as numerous as video borrowing from public libraries, are institutionally authorized through a different process than are public library holdings. The vast video libraries on YouTube and other video sharing sites typically have little institutional authorization.  Institutions are as much a part of reality as is human nature. Institutions change much more rapidly than human nature, but much more slowly than human behavior.  Public libraries as institutions almost surely will endure.  But the share of video works distributed through public libraries probably will increase greatly.

*  *  *  *  *

[*] These are rough, imperfect estimates, but they seem to be the best available data-based figures that currently exist.  Complaining about statistics is easy. Analyzing data and calculating statistics are difficult, but useful.

evolutionary origins of the iPhone

The evolutionary origin of the iPhone has recently been pushed back another million years. More specifically, scientists recently found evidence that hominini used tools about 3.4 million years ago.  Dr. Zeresenay Alemseged, Curator of Anthropology at the California Academy of Sciences, explains:

Tool use fundamentally altered the way our early ancestors interacted with nature, allowing them to eat new types of food and exploit new territories. It also led to tool making—a critical step in our evolutionary path that eventually enabled such advanced technologies as airplanes, MRI machines, and iPhones.

In related news, Android-based smartphones outsold iOS-based iPhones by 21% in the second quarter of 2010.  This is a major change from second quarter of 2009.  Evolution continues!

chest-thumping Android victory in the smartphone fight