OnePeople - Latest Comments

Re: Building a Timeline of Open Source in the US Government

Ted Benson — Mon, 09 Jan 2012 14:15:30 -0000

Hi Gunnar,

Thanks for trying out Datapress!

We think the Google Spreadsheet importer is choking on the colon in the spreadsheet name. On our copy of Datapress, the spreadsheet is imported, but the properties it finds in the spreadsheet are a bit odd, and some property names overlap with words in the spreadsheet title.

As a quick test, could you try renaming the spreadsheet to something with only alphabetic characters?

Also, feel free to follow up with us at datapress@csail.mit.edu and I can help debug further.

Re: Building a Timeline of Open Source in the US Government

David Karger — Sat, 07 Jan 2012 19:31:16 -0000

It looks like our latest release may have a bug; we just got a similar report from someone else. Will post an update in a couple days.

Re: Building a Timeline of Open Source in the US Government

Gunnar Hellekson — Sat, 07 Jan 2012 19:16:28 -0000

Thanks for the suggestion, David!

I loaded Datapress, but it seems to choke on the (presumably) well-formatted sheet here:

https://docs.google.com/spread...

Any suggestions? Where should I ask for help?

Thanks again!

Re: Building a Timeline of Open Source in the US Government

David Karger — Sat, 31 Dec 2011 17:37:25 -0000

It's nice to see you using our simile project timeline. If you'd like an easier approach, try our datapress extension for wordpress: http://projects.csail.mit.edu/... . It lets you set the timeline up using a wizard dialog with no coding. On top of that, by incorporation our exhibit tool (http://simile-widgets.org/exhi... it also lets you create maps, lists, and tile views, and facets for filtering the data on, for example, type of contribution. It's all served out of a google spreadsheet like you are doing now (or, you can upload your data to your wordpress installation). You can embed the timeline as you did or, for more space, place it in a pop-up lightbox.

One example site using datapress in production is here: http://www.quantnet.com/progra... . Besides a "when was it started" timeline it also shows a map of locations and a table of detailed data.

Re: A Common Criteria Primer

Wayman Smith — Sun, 11 Dec 2011 17:10:32 -0000

great article!

Re: A Common Criteria Primer

Martin Stadtler — Fri, 09 Dec 2011 14:21:45 -0000

Excellent write up, as is usual for Mr Hellekson.

Re: Brooklyn Eagle

fred — Sat, 19 Nov 2011 08:57:45 -0000

i'm looking for articles, pictures or any information on the Brooklyn Eagles semi pro baseball team from the late 1950's thru the 60's. Many of their games were played in the old Parade grounds. info to yeifred@aol.com please

Re: An Open Cloud Strategy, 3 Bullet Edition

tawster — Wed, 02 Nov 2011 04:10:09 -0000

Brilliant!

Re: An Open Cloud Strategy, 3 Bullet Edition

Guy Martin — Tue, 01 Nov 2011 19:23:47 -0000

Gunnar, is this where we are supposed to say: 'It's the people, stupid'? :D Great post!

Re: H.R. Giger + Giuseppe Archimbaldo Mashup

John Cleary — Sun, 10 Jul 2011 14:57:54 -0000

The Renaissance meets 3000 AD

Re: New York CIO Dr. Daniel Chan

Boingercat — Wed, 15 Jun 2011 19:53:38 -0000

Hi--I was doing an ancestry search for Alexander Samuel Rampell (he was my great grandfather), and ended up being directed here. Any relation? Thanks.

Lauren (Boingercat@aol.com)

Re: DOD Open Technology Development Guide Released!

Gunnar Hellekson — Tue, 07 Jun 2011 18:52:21 -0000

The source is now attached to the original post. There's still no permanent link at a .mil or .gov address, but it's also being hosted by the good people at Mil-OSS: http://mil-oss.org/resources/p...

Re: DOD Open Technology Development Guide Released!

Gunnar Hellekson — Wed, 18 May 2011 18:36:32 -0000

It hasn't been posted yet, which is why I'm hosting it here for now. As soon as there's a permanent link, I'll update the post.

Re: DOD Open Technology Development Guide Released!

Chris Williams — Wed, 18 May 2011 18:34:54 -0000

Do you have the original link for where you found this PDF?

Re: A Truly Open VistA

Hodgebrad — Wed, 13 Apr 2011 19:44:40 -0000

Cache is a new name for Mumps.

Re: A Truly Open VistA

Medicalquack — Wed, 13 Apr 2011 13:53:00 -0000

I agree the 2 should talk, why not. MUMPS is not all that bad as other major hospital EHR systems us a variated and updated version of it for their back end, like Epic:) the new name for old MUMPS slips my memory right now though.

http://ducknetweb.blogspot.com...

Back in 2008 a hospital rolled out Vista in record time that sat on the outskirts of Beverly Hills but nobody would finance them and they closed. It was state of the art too and many celebrities had care there with menus from Wolfgang Puck:)

http://ducknetweb.blogspot.com...

Re: The Hazards of Open Data Exceptionalism

sbma44 — Wed, 06 Apr 2011 15:34:23 -0000

I think you're absolutely right that there's fat to be trimmed from these efforts. After launch, in particular, it should be relatively cheap to maintain many of these sites on an ongoing basis. The data, storage and traffic needs should typically put maintenance efforts for a given site in the tens of thousands or low hundreds of thousands of dollars per year range -- not millions.

Of course, this isn't an ideal time to be addressing these issues. Finding efficiencies while leaving room for experimentation is important -- the equivalent of e-Gov projects committing to lose weight and get in shape. But the way to achieve that is clearly not by saying "we agree you should lose 50 lbs; you can pick what to cut off" -- which is what the originally proposed budget reduction amounted to.

Finally, I agree that going to the root of these systems is important: it's the only way to address some of the fundamental pathologies like poor data quality that make them less than fully useful. But I think it's a mistake to abandon integrated efforts. FPDS-NG is maintained by incumbent agencies and has its own points of brittleness and ossification. There can be real benefits to integration programs that are consumer-facing, and which work at a higher level of abstraction. It's a chance to try to reconcile disparate data models (e.g. grants and contracts) and to push the ball forward in ways that the old guard wouldn't even consider (e.g. disclosure of contract language). And it's a way to acknowledge that regulators, IGs and other oversight officials really do have different responsibilities and workflows than journalists and citizens.

Re: The Hazards of Open Data Exceptionalism

brianbehlendorf — Tue, 05 Apr 2011 04:23:42 -0000

Well said. I also think many of us can be ardent supporters of these efforts and still a bit mystified at the budgets required to keep them running. Perhaps it's due to the gauntlet of the federal contracting process, and the particular demands drove a lengthy and feature-rich development process. But one is tempted to say that if the tool/platform development had been conducted openly and resulted in reusable open source code, then the development costs would be lower and the costs to operate likely lower as well. What large company, NGO, or government would not want their own data.gov or IT Dashboard? It would also mean that a re-compete for the contract to operate a site like data.gov could be much more competitive. It's great that the IT Dashboard was finally released, but it'll take awhile before it achieves the kind of velocity and leverage that it could have achieved had it been open from day one.

Open data advocates need to argue as vociferously for open source code and dev processes as they do for open APIs and data sets. The latter is not long-term resilient without the former.

Brian

Re: Johanna Blakely on Free Culture, Fashion, and CentOS

Gunnar Hellekson — Sun, 03 Apr 2011 20:29:00 -0000

D'oh. Nice catch, Scott. Thanks for the kind words!

Re: Johanna Blakely on Free Culture, Fashion, and CentOS

Scott Dodson — Sun, 03 Apr 2011 20:26:21 -0000

Gunnar,

Small correction, she actually refers to books, film, and music as High IP industries. Good find and good write up!

Re: Google contacts in mutt and vim

tijptjik — Wed, 30 Mar 2011 19:48:25 -0000

in your .vimrc is says google-contacts-lookup.py, but your file is .sh , is that correct?

Re: Remembering Roger Lane

OonaShanley — Tue, 01 Feb 2011 01:38:46 -0000

Dear Lane Family,

I am a New Yorker and a fellow Columbia Alum from the Graduate School of Social Work. I remember the terrible tragedy of Jodie's heroic death as she tried to save her dogs. Even after 30 years in the City, I never realized the extent of the stray voltage problem. Thanks to Mr. Lane, I have access to the data he published and I am very grateful for his loving kindness to thousands of New Yorkers he did not personally know.

Please accept my condolence for his sad passing. He really was a hero too, turning unimaginable pain and suffering into a life saving remembrance of Jodie. I am very grateful for his work every time I walk my tiny dog outside when it rains or snows and I shall remember them both all my life.

Tonight they are both in my prayers as the snow falls.

Re: Remembering Roger Lane

Shep — Sun, 09 Jan 2011 11:52:00 -0000

Gunnar, this was touching, saddening, and enlightening. Thank you for sharing.

Re: The government doesn’t look good naked.

Tom Lee — Wed, 15 Sep 2010 19:00:21 -0000

Hi Steve -- thank you for engaging with our analysis in this level of detail! Having people dig into our work in the way you have is exactly what we were hoping would happen. I'll respond here, though if you'd like to move this conversation to one of our announcement posts so that more people can find it, I'd be happy to do so. Also, apologies for the delay in my response -- I needed a bit of time to get everything together.

You're right to single out Medicare/Medicaid as outliers. The situation here is a bit complicated: those programs don't report individual transactions to FAADS-PLUS/USASpending. Indeed, payments to individuals do not have to be reported under FFATA, presumably because of privacy concerns. However, in the past these programs *have* reported block payments aggregated at the county level to USASpending (and to FAADS, the separate system maintained by Census, where I believe they continue to be required to report). My understanding is that the reporting stopped because of a technical disagreement between HHS and OMB.

So this is money that doesn't legally *have* to be reported -- however, we know that it *can* be reported (it has been in the past). And we think it *should* be reported. As I'm sure you know, healthcare is a huge and growing portion of the federal budget. I don't think USASpending can be considered a useful picture of how the government spends money if health spending is excluded.

The situation is even *further* complicated by the fact that OMB seems to have resolved its disagreement with HHS and added the data in the past few weeks, after we received our snapshot of the data. Unfortunately, exporting data from the site remains difficult, though we have reason to believe the situation will improve soon. We certainly intend to re-run our analysis once the FY 2010 data is in.

Let me respond to your concerns about the prominence of the timeliness and completeness metrics. We think there are important components of making the data useful, but understand that others may disagree. However, they contribute a relatively small amount to the $1.3T figure, once you make sure that you're not double-counting any rows. I've asked the lead analyst on this project, Kaitlin Lee, to provide some more detailed breakouts explaining how we arrived at the $1.3T figure. You can find CSVs with this data on our resources page. There are a few things that I hope you will note. First, we feel that we've been quite generous to the government in our analysis. We only count reporting as being inconsistent if the CFDA and USASpending numbers disagree by more than fifty percent. We also extend a 50% cushion beyond the statutory requirement before we count a record as being late. We also exclude loan reporting from the $1.3T figure, because the numbers there are so large (and suffer from so many errors). And of course this number doesn't capture programs that aren't reported to either the CFDA or USASpending. Second, the $1.3T figure is arrived at by evaluating things at a program level, not an agency level. Because of the thresholds we use for determining compliance, summing things at one level doesn't give you the same amount of "bad reporting" as you'll get at a different level.

We feel that the $1.3T figure was arrived at fairly. But we also don't want anyone to get too hung up on it. It's been an important tool for drawing media attention to this problem, but we feel that our granular, program-level analysis is much more valuable, both because it reveals specific classes of data problems, and because it can help identify agencies that are managing their reporting properly. We think this analysis can be a real tool for government -- and that there are other things we can offer along those lines. Stay tuned for more on that.

One final note: all of our code is on Github (you can find that link on the resources page, too). I'm afraid the docs are a bit thin right now, but we'd be more than happy to go through our work with you if you're interested in diving in further.

Tom Lee
Director, Sunlight Labs

Re: The government doesn’t look good naked.

Dave Neary — Tue, 14 Sep 2010 10:44:15 -0000

This seems appropriate: http://sirhumphreyappleby2010....

"As time passes [politicians in government] realise they have more to lose than to gain from public knowledge of what they are up to. Each month increases their tally of catastrophic misjudgements, pathetic deceptions, humiliating retreats and squalid compromises. They very soon come to understand that sound and effective government is only possible if people do not know what you are doing."

(Just in case people think otherwise... that is satire)

Dave.