language technology for Paraguay

Earlier this month, we went to Paraguay. Why'd we do that?

Paraguay is the only country in the Americas where people are bilingual with a European language and the indigenous language. Paraguayans, the majority of them anyway, really do speak Guarani, or depending on context, a mixture of Spanish and Guarani called Jopará.

While we were there, we talked with Guarani-language teachers at the institutes where they're training translators and linguists. They let us sit in on some of the classes and talk with the students. We're working on building them a computer-assisted translation webapp that will help us collect lots of bilingual text! This is going to be huge.

While we were in the area, we also talked with the local One Laptop Per Child folks; there's an OLPC installation in the little town of Caacupé, which is near Asunción. The OLPC folks said that we should probably go visit local grade schools. Which we did!

It hit me, after the first visit to the schools, how much we were making use of our foreign-white-scientist privilege. We didn't have anything to do with the OLPC project -- aside from a desire to collaborate -- but here we were, wandering into schools without so much as a release form, talking to the kids. I'm trying to imagine Paraguayan scientists coming to the US to observe technology use among los niños estadounidenses.

The really interesting thing here: the kids in Caacupé weren't so surprised to see foreign scientist-looking guys coming to talk with them. They were really friendly, and eager to show off what they could do with the laptops! I get the impression this happens fairly frequently.

So there's a lot of stuff that needs to get built, to make computer use in Guarani more pleasant.

• At a very basic level, it's hard for people to type the diacritics that you need for Guarani, if your keyboard layout is set to Spanish. The diacritics for Guarani actually aren't that weird; they've got tildes on some vowels, but you see that in Portuguese too.
• There's no good spellchecker. Guarani morphology is pretty complicated, so this is not an easy thing to build. But we know a guy who's working on it...
• Text-to-speech. They kids in the schools have text-to-speech for Spanish, and they love it! There's a program on the OLPC where you can send messages to a friend's computer, and the receiving computer will speak your message. It's hilarious. But it doesn't work for Guarani. And as you get further out into the country, the kids are more likely to be monolingual Guarani speakers...
• The computer-assisted translation website: we're working on it. I'll write more about this soon...

I gave a talk about all of this at the computational linguistics seminar: here are the slides!

this is something new and beautiful: Coursera and Udacity

Just last week, I finished the coursework for Coursera's machine learning class. It was great! I had a really good time with it, and I'm fairly proud of the accomplishment.

If you've been within earshot of me in the past few months, you probably know that I'm really excited about Coursera and Udacity and their ilk (including, but not limited to, edX, Khan Academy, and Duolingo). There are two experiences I'd like to contrast with taking a course on Coursera.

Some years ago, I was living in Atlanta and working a real job. And I went over to the Georgia Tech math department to see about taking some masters-level statistics classes, imagining that they would let me pay them lots of money in exchange for taking classes at the university where I had just graduated months prior. But it turned out that they wouldn't let me do this without being admitted for a full-time degree program.

Fewer years ago, I was starting my PhD at Indiana, and knowing exactly what I was there to learn, I picked out three classes: one on NLP, a computational linguistics class (from the Linguistics department), and one from Stats. I got a mild hassle from the department about my choices: these were all "fun" classes, and shouldn't I work on fulfilling my breadth requirements? I've since finished my IU coursework, and let me say: not all the classes I had to take as a result were very interesting, or even very well taught. Some were downright bad.

But now there are free online courses that are meant to be good, such that you take the ones you're interested in taking, as opposed to expensive in-person courses that may not be good, but you're obliged to take them anyway -- this is huge.

Whether or not you think that teaching in person is going to stay relevant, not everybody has access to good teachers in person. This remains true even for people at universities.

Moreover, online classes lower the barriers to entering or leaving a course to almost nothing. Want to sign up for a class just to try it out? Nothing could be easier! Don't enjoy it, or it's not what you thought it was, or find out you're busy with other stuff? Nothing lost, try a different one! But if you stick it out and put in the effort, then not only have you learned something, but also you get a certificate that says you finished! (maybe these could be OpenBadges sooner or later...)

There are going to be lots of bytes spilled about these things in the coming years, but just to make it clear: I'm jazzed about helping people who want to learn things get access to material about those things. And the World Music class is starting up soon, which my mother and I are going to take! Because why not?

reading: Religious Literacy by Stephen Prothero

Not too long ago, I read the very thought-provoking Religious Literacy: What Every American Needs to Know and Doesn't, by Stephen Prothero, likely because it came recommended by Dale McGowan. The discussion of the history of religious education in the US is fantastic.

The main argument of the book is that, until recent decades, we knew quite a lot about Protestantism,  through instruction at home, in churches, and even in the school system. People apparently used the verb "catechize", as something you did to children. But this knowledge of what's actually in the Bible, and actual church doctrines is, these days, largely lost on us in the US, even though we're very caught up in our religious identities and church attendance is huge.

Political discourse is full of religious allusions, but we often don't get the references. I would have appreciated more examples of problems that this causes in practice -- is it really an issue for being a citizen in a democracy?

Prothero notes, early on, that Europeans are much less religious than Americans but more familiar with religious content. Dale McGowan says "faith is most easily sustained in ignorance"; knowing a thing or three about a few different religions makes it easier to not get caught up in any of them. Is the major problem caused by religious ignorance that it makes it easier for preachers and politicians to jerk people around by telling them that God says thus-and-such?

While Prothero doesn't address the question of why the more broadly-educated Europeans don't tend to be churchgoers, he does put forth a policy suggestion, that our curricula should have more information about the world's religions. And while I agree that it's probably a good idea, he doesn't say much about the sorts of changes we might see, with better religious studies education. I must admit, I have a hard time thinking of education-about-religion as anything except a strategy against the influence of seemingly-devout people.

Perhaps I'll pick up his more recent book, God Is Not One, about the fundamental disagreements between different religions, contrasting with the framing you'd get from Huston Smith or Karen Armstrong, who argue that different religions are grasping towards the same fundamental truth. I'm really curious about his personal position, because Prothero identifies himself as an Episcopalian, but hasn't thus far talked about any particular benefits of people believing any particular thing.

here come some explanatory examples

Having recovered from all that fruitcake and holiday cheer, I've started digging through the code I wrote over the past semester, looking for things that could be straightforward examples.

So far, I've got:

• calculating the entropy of a discrete random variable
• a cute implementation of finite-state automata with matrix multiplication
• calculating the probability of a Markov process going to a particular state, again with matrix multiplication
• a simple CYK-style chart parser for probabilistic grammars (computing inside probabilities, outside probabilities, and the most probable parse)
• a parse evaluator that gives precision and recall for parse trees
• probabilistic part-of-speech taggers that take into account bigrams, both by trying all combinations of tags for the words and using the Viterbi algorithm
• Some pretty clean code for hidden Markov models in general

I've already checked these in over on narorumo. They're all in Python, but some depend on nltk or numpy.

They'll be increasingly clean and documented over the next week or so. I hope these are helpful to somebody!

explanatory power of working examples

The NLP algorithms I've been studying since I started back at school aren't particularly complex. But they're often described with really dense notation: maybe your field does this too! Here's a description, for example, of how to calculate an "outside probability" -- it's the (joint) probability that a particular nonterminal symbol covers a certain chunk of text, and the words outside the span of that nonterminal. This is from Fei Xia's lecture slides (and I think these are pretty good).



Maybe what I need is more practice picking apart dense notation, but in all honesty I have trouble keeping track of what the different letters mean. Maybe a nice dynamic programming implementation springs to mind for people smarter than me, but I have to stare at it (and the surrounding slides) for quite a while!

I think I'd be making a pretty good contribution to the world if I took the algorithms I'm learning and wrote down the most straightforward pseudocode and prose versions I can, with a running Python implementation and descriptive variable names. Surely many people out there would find code easier to digest!

Somebody's already done precisely this with the Viterbi Algorithm wikipedia page, and I'm very grateful to that somebody.

my code has a side effect: learning!

So I'm working through the exercises in Real World Haskell. Here's what I've got so far. As of this writing, I'm in the middle of chapter 4. RWH totally deserves all the buzz it's been getting; it's very approachable and well-written, and my Haskell is improving rapidly. The book is still free online, but I bought the printed copy, in large part because Bryan O'Sullivan gave such a great talk at OSCON.

My earlier Haskell project had stalled out due to my not being very good at the language yet. What I want to do is build a compiler for Logo that produces JavaScript, with the turtle graphics on a canvas tag. Logo is an acceptable Lisp! I'd love to see it get more use.

Interestingly, though:

• There's already a beautiful Logo implementation that runs in javascript: papert.
• Mark is working on something like papert, but for Processing -- he wants to put your Processing sketches, how you say, in the cloud.
  
• There are some really nice (and free!) introductory CS texts that feature Logo. Brian Harvey has a pretty serious three-volume set. And there's a version of How to Think Like a Computer Scientist done in Logo.
  

reconsidering applets

Applets are a really tempting idea. The JVM is a serious piece of machinery; despite its startup time, once it gets rolling, Java can go fast indeed. And although it hasn't been very widely used, applets and javascript have had a means of calling back and forth for quite a while. It's called LiveConnect. (although LiveConnect may be replaced at some point...)

So, a beefy, fast VM that runs inside the browser, coupled with a nice web UI, and no manual installs for your users -- big win? Maybe! On considering this, one of my first thoughts was to put Jython into an applet and build something akin to JES -- a little Jython IDE in the browser! The kids would love it! So I set out to make this happen, or at least build a proof-of-concept.

I ran into a serious snag, though -- the Jython interpreter works by compiling to bytecode and bringing it up with the class loader, the latter of which applets aren't typically allowed to do. Special permission can be granted with a Java policy file, but that's a very high barrier to entry. It's certainly not the sort of thing a person might do on a public terminal. Signing the applets may also be an option -- but that's also a pain, and anecdotally, I think signed applets may not always get the same rights, cross-platform.

Once I understood the permissions issue, I did get a little Jython repl going -- it just hooks "eval" up to some javascript for output. Nothing fancy, but kind of satisfying. For now, though, it doesn't look like Jython IDEs are coming to the browser. Until we rewrite it to not use of the class loader. Or just use some more modern approach.

Some other people have had similar thoughts!

• Interactive Jython Console: this one, for the exact same reasons that mine does, requires a java policy file.
• the ruby-in-browser project. It does exactly pretty much what it sounds like it does, also by means of an applet and LiveConnect. Interestingly, JRuby doesn't seem to use the class loader, it just works! Check out the demo! [thanks for the heads-up, Lindsey Kuper!]

possible upcoming projects

I need a cool side project. I have a bunch of ideas that I think would be interesting: here they are. I've already started futzing around with a few of these... comments or suggestions are definitely welcome.

New Things To Build
- Sketch out and build an online community where kids can make stuff in code and share it with each other, like MOOSE Crossing only awesome and on the web and easy to use and linkable. Does this want to be made of Scratch, or similar to it? Can we include proper inheritance? Make it work for the OLPC.

- TEB. Make TEB awesome. Does TEB want to use/be part of/merge with gnoetry? Related: That horoscope remixer thing that never worked right.

- Some automated way to calculate Erdos numbers, probably with the help of DBLP.

- A Scrabble bot. I've been thinking about this a lot, actually, looking at different algorithms for permuting strings -- but I think I can do Scrabble without looking at permutations... maybe all you really care about is whether certain groups of letters constitute the same bag. If it turns out we don't have to permute things (in n!), then we can probably plan several turns ahead. This might be similar to playing Backgammon,, but maybe we can't do something like that kind of policy learning; your set of possible actions changes so much every turn, and you'd have to use probability estimates to look into future turns...

Interesting Exercises, way already done by other people
- Build a language and virtual machine. Educational for me, not very useful for other people. Recommended by Strick. Building a VM at a pretty high level of abstraction might not be that hard, just think about stack frames and returning things. At what level do things like the Python VM work? The JVM? Surely there are different approaches used for this; what are they? Would it be hard to build something with multithreading in mind from the bottom up? What about a purely functional lambda-calculus implementation?

- Write a checkers bot. Tree search, AB pruning, etc. are pretty well understood, even by me -- the hard thing would be an evaluation function.

- Build a MUD, or at least a MUD framework. But in Scheme. ynniv and I somehow never got around to that...

- Another Sudoku solver. But in Haskell.

this blog post: for you, $50.

I recently met a fellow who's working on a doctorate in literature, but his previous background is in Library Science^W. I wasn't sure what the interesting problems in library science might be, so I wandered over to the wikipedia article and started falling through the links.

A few links out, I ran into the Serials Crisis article. Apparently (and Wikipedia articles close to the "Library Science" one are never wrong), the costs of subscribing to scholarly journals keep on going up -- libraries only have so much money for subscriptions, but there are ever-more academics and subfields, thus more journals. And if a given library cancels its subscription from a particular journal, that publisher's fixed costs are still fixed, so prices increase for the remaining subscribers.

The traditional academic journal system had seemed pretty shaky, especially in light of the Web; upsetting publisher websites (Springer, ACM Portal, IEEE's site...) seem like their sole purpose is to keep the enterprising students from reading an article. In light of how most of the science behind the articles is publicly funded in the first place, the articles seem like they should be public as well.

I wouldn't mind seeing companies like Springer just going away; universities seem totally capable of hosting journals -- over the web especially! There may be some compelling reason for the current system, and I'll try to find it out... but for the short term, tools like Google Scholar could go a little further out of their way to help us find the full text of an article!

Also:
http://en.wikipedia.org/wiki/Open_access
http://en.wikipedia.org/wiki/Open_access_journal
http://en.wikipedia.org/wiki/Open_access_publishing
The Serials Crisis: A White Paper for the UNC-Chapel Hill Scholarly Communications Convocation
The Crisis in Scholarly Publishing

(ping)

It's been a while since I've last posted: I should make an effort to have exciting technical things to say more often.

But! In the last few months, I've graduated, got a paper accepted to a conference, taught six weeks of summer camps to enthusiastic middle- and highschool kids, and made all the preparations to start up work with the Goog. My first day is Monday.

Topics that I've been looking into and will hopefully post about soon:
- Haskell -- maybe one day I'll have a more consistent opinion about how I feel about static vs. dynamic typing. Writing here on the issue will probably help sort it out. Or just building something big with Haskell.
- Statistics. I went out and bought All of Statistics and I've been thumbing through it a bit.
- CS Education and edutech. Good goodness, education. I've spent most of my summer with the childrens, trying out different ways of convincing them they want to know what I think they should know. It's been going well, for the most part.
- GWT and associated topics in Javascript and Ajax. I'm joining the GWT team in just a few days, so I'm working pretty hard on learning it!

more exciting edutech!

Scratch, from the Lifelong Kindergarten group at MIT, is this lovely environment for kids (or other novice programmers) where you can make funky animations and play sounds and do cool effects! The cool effects are a major selling point -- you can get funny animations right off the bat. Everybody loves a fisheye effect. And it has this lovely website, where kids can share and tag their projects! Web 2.0 ahoy!

It's really easy to figure out, especially if you've seen things like the LEGO mindstorms interface, or Alice -- commands snap together with a familiar building-blocks metaphor. This is to say that there's a fairly standard vocabulary for childrens programming environments, these days...

The Scratch intro "Facilitorial" video is here.

Also fairly interesting (and brand new on my radar as of today), is Greenfoot, which is another educational programming environment, perhaps for slightly older kids. It makes it easy to do simulations with different kinds of "actors" on these nice 2D worlds (they can be grid-worlds, but they don't have to be)... although you have to write some Java, it looks like, to build up your new behaviors. Maybe this is awesome too.

retrocomputing from the other side of the pond

Here's what I found out on my recent voyage through the wikipedias! Semi-vicarious nostalgia ahoy!

In the early 1980s, the BBC started an initiative called the BBC Computer Literacy Project, a major part of which was the production of the BBC Micro, a machine produced by Acorn Computers, complete with its own line of peripherals including expandable memory and various pluggable co-processors. There was an associated television show, The Computer Programme, which ran in various incarnations through the decade and featured music from Kraftwerk. The computers came with BBC BASIC, a rather more advanced system than the BASICs that were shipping stateside -- it had proper named subroutines and if/then/else, features most users on the MS-DOS side of things wouldn't see until QBASIC.

The mind-blowing part of the project was Telesoftware, whereby computer programs were sent embedded in the broadcast television signal, using Teletext, which is how the closed-captioning data was sent in Britain at the time. Analogue technology like broadcast TV feels so alien these days... but the Beeb was busy using it to send example programs to eager learners at home.

There seems to be a pretty active online community of BBC/Acorn enthusiasts out there, two and a half decades later.

You know how to use the Googles, of course, but here's another, more detailed overview of the BBC/Acorn system.

JES Prerelease!

I packaged up the next release of JES, our "Media Computation" environment for helping beginning programmers today. Honestly, if I was teaching kids about programming, I'd probably use this. It's a lot faster than the previous versions, it has a better layout and clever integrated help, it has LOGO turtles!, and like always -- it's really good for doing funky stuff to pictures. (gallery 1, gallery 2).

Fresh code! Here!

Any feedback would be greatly appreciated!
(slides for cs1315, the course where we make use of JES can be found here, in case you want to play around with JES but don't know what it's good for yet!)

Beautiful educational software :)

So many of you are probably aware that my job (ie: research assistantship, ie: how I'm getting gradschool paid for) is to work on JES, the Jython Environment for Students -- it's the little IDE that we use at Georgia Tech to teach introductory CS classes to non-majors and for summer camps as part of our "media computation" effort. A number of other schools use our stuff too, which makes me pretty happy. We're releasing a new version soon, and it's going to rock hard and be faster and include LOGO-style turtles! I'll put up links when it's out.

But speaking of LOGO-style turtles and media computation:
- The KDE Project has KTurtle out now, which looks pretty cool. It's part of their bigger KDE Edutainment effort.
- TuxPaint is also pretty cool! My friend (and noted cartoonist) Brett finds it pretty amusing. It reminds me of KidPix, which you might remember from your childhood if you're about as old as me -- and that's still out there, apparently?
- Tux Paint comes to us from New Breed Software, which has a bunch of other cool educational products.
- And if you only click one link on this post: Pictures Kids Made With TuxPaint

And, for completeness:
- Alice is a programmable 3D environment up outta CMU that rocks harder than I can describe in a quick blog post. As they put it on their site, "Alice will make programming a means to an exciting end." That exciting end is really easy 3D storytelling.

Red pandas and rhinoceri and turtles? ...

Why is there not a client-side javascript version of LOGO?

How cool would that be? Turtles in your browser? ... Small children having access to Papertian goodness wherever the Internets are tubin'?