Smart Meters, Electricity Usage, and Privacy

My current apartment building has smart meters that measure and report each unit’s electricity usage. While you can get real time high-resolution usage data if you’re willing to stand in front of the meter, the utility company also provides daily usage data through their website.

I got tired of logging in every day, and the visualizations they provide are a little clunky, so I wrote my own Python script to automatically log in, download the data, and graph it using Plotly. The code is available on GitHub, though it’s specific to my utility company (United Illuminating). doesn’t allow arbitrary embeds, so below is a static image of the Plotly output. Here’s a live version that’s updated every 24 hours. It has mean lines and trend lines (simple linear regression) by month. I also built an Android app for my phone that fetches a vertical layout of this graph in a WebView.


I went to a talk this past week by Elizabeth Wilson, an associate professor of Energy and Environment Policy at the University of Minnesota. It was a really interesting talk that covered a lot of different topics, including some of the privacy implications (and perceptions) of smart meters.

One of her slides contained the following image, which documents electricity demand in a British household:

Screen Shot 2014-11-23 at 11.20.29 AM

I found this fascinating; just a few weeks earlier, I had been discussing with a friend visiting from out-of-town how much usage one could infer from reasonable resolution electricity usage data. I was skeptical about how much we could really learn.

  • My day-interval data can probably give you an idea of whether I’m home or not, although it might not be any better than just distinguishing between weekdays and weekends. You also might guess when I turned on the heat for the winter.
  • This project uses the rtl-sdr dongle to decode the messages sent on the 900 MHz ISM band by a particular brand of smart meter, and receives data every few minutes. I would ultimately like to do this for my brand of meter, which, AFAICT, is broadcasting on the 2.4 GHz ISM band every two minutes or so.
  • The graph above uses minute-interval data, derived not from a smart meter, but from a data logger installed at the customer’s home.
  • Proof-of-concept research shows that with two-second interval data, it’s possible to determine what television programs are being viewed (assuming you generate fingerprints of viewable content ahead of time).

The graph above has found its way into presentations, papers, and even anti-smart-meter websites, often as an example of how smart meters can reduce privacy and reveal significant details about personal habits.

Curious, I tracked down the original source of the graph. It comes from a paper published in 1999, which tracked the electricity usage habits of 30 British households. But the methodology section contains some critical details that are always omitted when this graph is bandied about:

Questionnaires were designed for completion at the time the logger was installed in order to identify the primary characteristics of the household and in particular which types of appliance were fitted. Some householders also completed tick-box ‘appliance utilisation’ diaries for one weekday and one weekend day during the logging period; these served to improve our ability to ‘read’ the demand profiles… User behaviour is generally deducible from the database of monitored electricity demand profiles and the knowledge of a surveyed household’s appliance stock… the most apparent classification was by periods of occupation.

In other words, the researchers not only had access to minute-interval data about demand, they also knew what appliances people owned, and in some cases knew when people were using these appliances—all because people volunteered this information. The researchers also logged power factor data, and the internal and external temperatures at the home being monitored.

Note the numerous errors on this Stop Smart Meters website, which includes the graph:

  • They call it a “hypothetical graph” (it’s a real graph, from a real research paper)
  • They imply it was generated by a current-era smart meter (it was generated by a data logger installed at the consumer’s home, which consisted of a 90’s era Acorn PC, and which measured several other variables)
  • They claim that the appliance labels in the graph are based on “signatures” derived from variation in power factors (in fact, the original paper notes that power factors are almost always close to 1, except for the refrigerator and washing machine in spin mode; and they omit that these households answered questionnaires and completed diaries about appliance usage)

Without knowing details about appliance ownership, it would be far harder to determine what is going on in household usage. For example, I suspect there is little way to distinguish between an electric kettle and a toaster based solely on usage, other than to guess that one might make tea before one makes toast.

Certainly, some things will always be obvious (you have a refrigerator, you cook using an oven); but others are less transparent. I have an unusual appliance that pulls 1.5 kW for about 90 seconds a few times a day (it’s not a kettle), and I doubt anyone at the utility company will ever guess what it is.

Consumers should realize that smart meters can benefit them. Apart from lowering costs for the utility company and making them more aware about outages, knowing how much you’re using can help you save money. Where I live, electricity costs about $0.21 per kWh, so savings really add up. Utility companies can help by making sure consumers have access to their data. I wish my company offered higher resolution data, but even the day-interval data has been very informative.

Posted in General | Tagged , , , , | 4 Comments

TPP Chord Diagrams using Circos

I made some chord diagrams based on the new TPP intellectual property chapter draft, leaked last Thursday. These diagrams show the extent to which TPP parties band together on bracketed text in the May 16 draft, discussed in more detail here.

The width of the outer country band shows the total number of times a country joins with other countries in the draft. The links emanating from the country and joining it to others vary in width based on the number of times those two countries are paired in the text.

All these images are licensed CC BY. Please feel free to use them (click for higher resolution versions) with attribution to Gabriel J. Michael / The images were produced using Circos, and code to extract the raw data from the TPP draft and generate the Circos data files can be found on GitHub.

I also threw together a series of HTML image maps so that you can click on a country’s band, and its links will be highlighted.


Posted in General | Tagged , , , , | Leave a comment

New leaks show US gaining ground in TPP IP chapter

Nearly a year ago, Wikileaks released a draft text of the Trans-Pacific Partnership’s intellectual property chapter. I did some visualizations and analysis in the wake of that leak, which you can find here.

Today, Wikileaks released a second draft text of the same chapter. The text of the first leak dates from August 30, 2013; the text of this leak dates from May 16, 2014. While others are analyzing the content of this new leak (see Knowledge Ecology International, Public Citizen, Concurring Opinions, and the EFF), I decided to take a look at how negotiating positions have changed in the nearly 9 months between the two draft texts.

The figure below shows how countries positions have converged between the first and second leaked drafts. Each circled country code indicates where the country stood in the first draft; the arrows emanating from the circles indicate the direction and amount of convergence, and the end of the arrow indicates where the country stands in the second draft.

TPP IP Chapter Negotiating Position Changes

These movements take place around a central point, indicated by the large black star in the middle of the graph. In order for the agreement to conclude, all parties will eventually need to reach this central point.

This next figure is a partial inset from above; it shows the dashed dark gray rectangle from above in more detail, but only shows positions from the second draft.

TPP IP Chapter Negotiating Positions, May 16, 2014

My approach visually indicates how often countries end up on the same side of bracketed text in the drafts. Thus, a large degree of movement doesn’t necessarily indicate that a country changed its views on an issue; rather, it means that is joined by significantly different (and often more) parties than in the earlier draft.

In fact, given the analyses conducted by others, the significant convergence in the second draft suggests the United States is succeeding in convincing other TPP parties to join its positions.

As we might expect in a negotiating process, there’s a significant amount of convergence over time. However, even though there is significantly less disagreement between all parties in the second draft, the United States still lies farthest from the central point of agreement.

I constructed these graphs by extracting country codes from the leaked drafts, counting the frequency of dyadic relationships between countries, then creating distance matrices and plotting these as graphs using multi-dimensional scaling. You can find some very rough code on GitHub.

This post and the included images are licensed CC BY-SA 4.0, and may be shared and reposted with attribution. When reposting, please include a link back to this page, which will contain the most up-to-date version.

Posted in General | Tagged , , , , | 1 Comment

Can you copyright a tweet?

Gabriel J. Michael / gmichael at gwu dot edu

Back in January, I had a brief exchange on Twitter prompted by this news story. The gist is that A. O. Scott, film critic for The New York Times, posted a tweet about the film Inside Llewyn Davis. The film’s promoters took out a full page ad in the Times displaying the tweet (or more accurately, the last two sentences of the tweet).

The linked article’s discussion assumes that Scott “own[s] the copyright to his tweets,” but notes that by tweeting, Scott could be presumed to be granting an implied license for reuse of the tweet elsewhere.

But can you even copyright a tweet? I did some research and was unable to come up with a clear answer. There was some academic discussion of the issue, and occasional instances in which Twitter users claimed others were infringing their tweets, but I could not find a clear instance in which someone had actually registered a copyright in a tweet.

So, 7 months and $35 later, I have my answer: no, you cannot copyright a tweet.

That, at least, is what the registration specialist at the Copyright Office decided to send me in response to my attempt to register this tweet as a literary work entitled “Tweet #452″:

Monkey bar fallacy: a bad person using something makes it bad. E.g., users of monkey bars include: children, TERRORISTS #tor

Of course, the rejection of this particular tweet does not imply that no tweet can be copyrighted. Perhaps the registration specialist did not feel my tweet was valuable or creative enough, and thus did not pass the (very low) threshold of originality.

This makes me wonder whether short poems like haikus are eligible for copyright protection. Browsing the Copyright Office’s registration database, I can find a number of registered literary works labeled “haikus” that are no longer than one page. Perhaps I would have had more luck if I had instead tweeted a haiku:

Monkey bar fallacy:
A bad person using something
Makes it bad.

(For sticklers, yes, I know it’s not 5-7-5, but it is 17 syllables.)

Ultimately, I wonder if the Copyright Office applies more scrutiny to short literary works than it does to photographs. In the U.S., we work under the assumption that every photograph taken by a human being is copyrighted. But I take a lot of photos, and many of them take far less time, effort, and creativity to compose than a tweet. Here’s an example:

Greek YogurtOther countries have found that some photographs simply aren’t creative enough to warrant copyright protection. Wikipedia has a brief summary and link to the German text of a Swiss case in which a reporter’s photograph of a man holding record books was ruled ineligible for copyright.

It would be a fun, albeit expensive, experiment to try and register a variety of liminal works: handfuls of sentences, short quines, run of the mill photographs, “sculptures” made of a few Lego pieces, etc. I would contest the office’s decision about my tweet, but I don’t want to pay $250 out of pocket to do so, and I also don’t really want to write a funding proposal to try and convince someone else to give me the money.

To wrap up this little experiment, the Copyright Office’s online registration process allows registrants to submit comments with their registrations. I submitted the following text, although I have no way of knowing whether it was ever read:

In Ashleigh Brilliant v. W.B. Productions, Inc. (Civ. No. 79-1893-MBM, S.D. Cal Oct. 22, 1979), a U.S. District Court found that Brilliant’s copyrights on three epigrams were valid and enforceable. The epigrams were 12, 15, and 10 words respectively. Each was a single sentence, lacking rhyme or meter; rather, their originality consists of their pithiness. Tweet #452’s originality is similar: using 20 words and two sentences, it exposes the logical fallacy inherent in blaming tools, using humor and topical examples to communicate the point.

Circular 34 states that “copyright law does not protect names, titles, or short phrases or expressions.” Tweet #452 is clearly not a name or title, leaving only the question of whether it is a short phrase or short expression. The Oxford English Dictionary defines a phrase as “a small group or collocation of words expressing a single notion, or entering with some degree of unity into the structure of a sentence; a common or idiomatic expression.” Expression is similarly defined as “A word, phrase, or form of speech.”

Tweet #452 cannot be classified as either a “phrase” or “expression,” since it contain two complete sentences (i.e., two subject-verb pairs). Thus, Tweet #452 does not fall within the scope of Circular 34. Even if Tweet #452 were considered a phrase or expression, phrases or expressions as such are not necessarily ineligible for copyright, since Circular 34 specifies that only “short phrases or expressions” are ineligible for copyright, thereby suggesting that longer phrases or expressions are eligible.

Update (August 4): I’ve had a few requests to post the correspondence I received from the Copyright Office, so here it is.

This post and the included image are licensed CC BY-SA 4.0, and may be shared and reposted with attribution. When reposting, please include a link back to this page, which will contain the most up-to-date version.

Posted in General | Tagged , , , , | 1 Comment

Visualizing TISA Negotiations

Gabriel J. Michael / gmichael at gwu dot edu

Yesterday, Wikileaks released a draft text of the financial services chapter from the Trade in Services Agreement. I’m not going to get into the substance of the treaty or chapter, but as with previous leaks, this one contains information about country positions.

As I’ve done before, I extracted information about country positions using Perl/regular expressions, then graphed the resulted information in R using the igraph package, resulting in the following network graph:

TISA Network GraphThis graph excludes many TISA negotiating parties, as many parties did not appear in the draft text. As you can see, there is a strong link between the United States and the European Union, and both also have proposed a significant number of provisions not joined by any other party. Panama, a major financial hub of the Americas, also makes a large number of proposals not supported by others.

The following table reports the frequency of country dyads (instances in which two countries both support the same bracketed text) in the leaked draft. It includes identities (i.e., dyads between a country and itself), which represent the number of times a country makes a proposal not supported by any other country.

The country codes should be fairly obvious, but just in case:

PA = Panama, US = United States, EU = European Union, KR = South Korea, AU = Australia, NO = Norway, TR = Turkey, CH = Switzerland, HKC = Hong Kong, CA = Canada

Country 1 Country 2 Frequency
PA PA 16
US US 15
US EU 14

This post and the included image are licensed CC BY-SA 4.0, and may be shared and reposted with attribution. When reposting, please include a link back to this page, which will contain the most up-to-date version.

Posted in General | Tagged , , , , , , , | Leave a comment

GIPC guest post on Fair Use fails Copyright Law 101

Gabriel J. Michael / gmichael at gwu dot edu

In another instance of the U.S. Chamber of Commerce’s Global IP Center (GIPC) posting factually incorrect information, on June 11, the GIPC cross-posted a blog post from Plagiarism Today purporting to clarify the meanings of “5 Copyright Terms We Need to Stop Using Incorrectly.”

I ran across this post because of the following GIPC tweet (screenshot posted here in case it mysteriously disappears):

GIPC Fair Use TweetUnfortunately, the linked post gets basic facts about copyright law wrong. On fair use, it reads:

What it Means: A fair use of a work is an infringement of a work where the court has determined that the infringer is not liable due to the nature of the infringement being within the bounds of the law.

I’m honestly not sure how much more wrong one can get. 17 U.S.C. § 107 plainly states:

the fair use of a copyrighted work, including such use by reproduction in copies or phonorecords or by any other means specified by that section, for purposes such as criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research, is not an infringement of copyright.

To say a fair use is an infringement is to fundamentally misunderstand the nature of U.S. copyright law. It’s also nonsensical. By definition, an “infringement” means going beyond what is permitted by law. An “infringement being within the bounds of the law” is no infringement at all.

The author’s confusion probably stems from the fact that in lawsuits, fair use is raised as an “affirmative defense.” Many people wrongly interpret this to mean that fair use is a kind of acceptable copyright infringement. This is incorrect. Wikipedia explains it nicely:

“Affirmative defense” is simply a term of art from litigation reflecting the timing in which the defense is raised. It does not distinguish between “rights” and “defenses”, and so it does not characterize the substance of the defendant’s actions as “not a right but a defense”.

In other words, an affirmative defense is an admission that you committed the alleged acts, but that other circumstances make these acts lawful.

GIPC blog contributors might want to take an introductory copyright law class before trying to educate others on the meaning of fair use.

This post is licensed CC BY-SA 4.0, and may be shared and reposted with attribution. Please include a link back to this page, which will contain the most up-to-date version.

Posted in General | Tagged , , , , | 1 Comment

GIPC’s misinformation on plain packaging

Gabriel J. Michael / gmichael at gwu dot edu

Protip for the U.S. Chamber of Commerce: it’s generally not a good idea to support your policy positions by pointing to the hardships of one of the most reviled industries on the planet.

The Chamber’s Global Intellectual Property Center (GIPC) recently published a blog post entitled “Plain Packaging: Keep Your Hands Off My Doritos,” but a more accurate title would have been “Keep Your Hands Off My Cigarettes.”

The blog post warns that plain packaging—a health and regulatory policy applied to tobacco products, designed to render them less attractive to buyers—would turn “grocery store shelves into an unnerving art gallery of morbid graphic images, or even the opposite—a brandless Orwellian muddle of boxes and bottles.” Given that plain packaging currently only exists in one country (Australia) for one type of product (tobacco), this warning has no basis in reality.

The author purports to be concerned about “recent reports coming out of New Zealand (and to some extent, California) about potential new restrictions on the use of trademarks for many of our favorite snacks and sodas due to health concerns.” Oddly, as of the time of writing, the New Zealand link provided was broken, and the California link makes no mention of trademarks, instead discussing warning labels.

It’s also hard to understand how “plain packaging takes away choice from consumers.” Consumers generally buy products for their content, not for the packaging (unless it’s Tiffany!). Packaging may drive interest or demand for a particular product, but regulating packaging doesn’t actually prevent anyone from purchasing the product.

The author dismisses Australia’s ground-breaking plain packaging law as having produced “mixed, if not skeptical results.” She includes a link to an article reporting that cigarette sales actually increased in the wake of plain packaging.

It’s tough to expect objectivity from a Rupert Murdoch property whose first sentence begins “Labor’s nanny state push…” as the linked article does. What follows is a hack job full of misinformation and fallacies. It turns out the “increase” in cigarette sales was just 0.3%. The article then engages in numerical deception, comparing this figure to a 15.6% decrease—over the previous four years.

In fact, an “increase” in sales of 0.3%, when compared to Australia’s population growth of 1 to 2% in recent years, actually results in a per capita decline in sales. Furthermore, the linked article provides no information about changes in the sales of other tobacco products, such as cigars, snuff, or chew, even though the plain packaging law also applies to these. The article makes much of the fact that in the wake of the law, consumers have shifted to lower priced products, but this is precisely the effect prior research on plain packaging predicted.

Furthermore, upon closer examination, it appears the linked article, published in The Australian, is simply false. Stephen Koukoulas explains:

The figures from the ABS show that total consumption of tobacco and cigarettes in the March quarter 2014 is the lowest ever recorded – and this with the series starting in 1959… Making this record low consumption of tobacco all the more fantastic is that the fact that the consumption numbers are not adjusted for population growth which, by definition, means per capita consumption of tobacco and cigarettes is also at a record low.

Making a mockery of The Australian’s story is the fact that, in seasonally adjusted volume terms, consumption of tobacco is 5.3 per cent lower in the March quarter 2014 than in the December quarter 2012 when the plain packaging laws were introduced.

The Chamber’s blog post references the “theory… that plain packaging… can deter consumers from consuming ‘unhealthy’ products.” First, cigarettes are not “unhealthy,” in scare quotes, they are unhealthy, period. Second, this “theory” is backed up by dozens of peer-reviewed studies and controlled experiments. Here’s one open-access article published in Tobacco Control that anyone can read. The GIPC, like many intellectual property lobbies, probably isn’t interested in evidence-based policymaking, but it’s never too late to start.

By the way, businesses don’t really need government help in turning “grocery store shelves into an unnerving art gallery of morbid graphic images.” They’re doing a great job by themselves.

This post is licensed CC BY-SA 4.0, and may be shared and reposted with attribution. Please include a link back to this page, which will contain the most up-to-date version.

Posted in General | Tagged , , , , , , | Leave a comment