Android: Piggy-backing on the WebKit browser

I've been looking for ways to not store or request username/password combinations for an Android app. The application I'm developing basically needs to get some data from a Google AppEngine application that is deployed, but that app uses authenticated users for doing some queries. I am actually reluctant to request username/password for the application to login, since I don't want people to even believe the combinations can be misused.

Ideally, I want the users to log in using some login provider on the android platform, or reuse login capabilities for gmail/calendar synchronization services, but I don't think that may be possible.

The objective is to just sync some data from the web, after the user logged in with their credentials. The login is then used to perform the query with. All this without needing to have another app register precious login details in its own space, which may get lost or recovered somehow by others.

Webkit to the rescue!

WebKit on Android can be used to show HTML help pages that were bundled along with the app. Also, you can programmatically construct a browser view then use events on the webkit browser to detect when a user finished its conversation (the login process) and then recover the cookies and use that to call the actual service with. The following snippest show how this can be used in practice:

WebView webview;

private class HelloWebViewClient extends WebViewClient {
@Override
public boolean shouldOverrideUrlLoading(WebView view, String url){
view.loadUrl(url);
return true;
}
@Override
public void onPageFinished(WebView view, String url) {
super.onPageFinished(view, url);
CookieManager mgr = CookieManager.getInstance();
Log.i( "HelloAndroid", url );
Log.i( "HelloAndroid", mgr.getCookie( url ) );
}
}

@Override
public void onCreate(Bundle icicle) {
super.onCreate(icicle);
setContentView(R.layout.main);

webview = (WebView) findViewById(R.id.webview);
webview.getSettings().setJavaScriptEnabled(true);
webview.setWebViewClient(new HelloWebViewClient());
webview.loadUrl("http://yourapp.appspot.com");
}

That's a good start for temporarily using the browser to do the login. The advantage is that username/password combinations never travel through your app and the standard web mechanism is used to obtain a set of cookies to use for navigating further. The username/password remain very volatile in this entire process.

Since I am sort of running the application in a browser, there are ways for this android app to 'step out' of the browser through one of the events and call local services. As a matter of fact, there is a very nice addJavascriptInterface call, which can register a specific interface in Javascript, which can be called by downloaded HTML pages. Certainly, care must be taken security-wise not to have any other pages loaded in the same browser instance that could also call the javascript interface.

class Logger
{
public void log(String content)
{
Log.i( "HelloAndroid", url );
}
}

..... setWebViewClient(new HelloWebViewClient());
webview.addJavascriptInterface(new Logger(), "Logger");
webview.loadUrl( ........

If then your remote HTML file contains:


window.Logger.log( 'hello Android!' );


You basically have a way to let your android phone do things through the remote server app. The WebViewClient should probably contain a couple of safety controls to disallow browsing to foreign destinations.

Why is this so interesting? because you can now create a server-based GWT app and create a nice-looking browser-based interface (generated or stored server-side) and possibly avoid a good deal of code for native interfaces.

It should even be possible to finish up the browsing part if the server dictates so and continue with some intent on the handset device (and possibly return later).

My intention is to just sync some information for another app and then use that data for other purposes locally. The browser allows me to use GWT all the way, so I can use that to do the very effective GWT-RPC calls to communicate data across effectively. The GWT client code can then call the pre-meditated interfaces in the android app, and we're done!

On towards the next phases... :)

Confabulation and how it relates to SVD?

I've been reading up on this confabulation theory. It's pretty interesting work, the theory, but the practical implementation isn't as nice as the ideas behind the theory :). The use of matrices in the implementation seems to basically make this a problem of statistical analysis. The difference with Bayesian theories and other probability frameworks is that this theory assumes a "causal" relationship between one and the other, not in the way that Bayes does, but through cogency maximalization.

In the end though, and reading the word "matrices" somewhere inbetween, it all sounds a bit like singular value decomposition with a twist, rather than something entirely new. I've been looking for ways to replicate their results somehow. I came up with the following:

• Using Hadoop as an underlying platform for simple tasks (like counting words, ordering them, etc.), you take out a lot of complexity out of a program. So I'm using Hadoop to count word frequencies and the frequencies that words are seen together and at which distance.
• Hadoop provides a platform where I'm mostly using files, intermediate results stored in files and reducers to bring back some sanity. If I'd had to program that myself, I would be more concerned about stability and reliability than the core of this little research, statistical analysis (in short).
• The results are used to train a couple of SVD's á la Netflix - gradient descent. Because I've got a good 1GB file of frequencies, I don't need to process text anymore, it's all ready for learning. (text processing required 5hrs to process and get the results together. The problem in learning is that the statistical significance is only apparent after the entire lot was processed, or you'd have to heuristically make estimates).
  
• The SVD's are about 60M in size. On my machine with memory left, I could theoretically get to 33 SVD's, working together.
• The pre-processed files allow me to process the entire "frequency" file line by line, one after the other. I post-processed it to take out any words I did not recognize.
• Since I know the statistical significance of each word, how it relates to another and by what distance, I can just keep on training the features of each SVD. Not knowing that beforehand makes training slightly difficult (doh!).
  

I also thought about possible results of confabulation if it were implemented as a chat bot. Suppose that the technology were ready to produce sentences as if a chat session was going on. The Turing test is one important test, where judges need to make clear distinctions between humans and computers. Since this is a subjective test, I can't help but think that there's also an element of expectation involved. The judges know what a computer does, how it operates and what its limitations are. Now, if someone invents something really cool that breaks that barrier, the judges may temporarily be baffled and come to incorrect conclusions. But as soon as they learn about the research, they may adjust their expectations, raising the bar for the future.

In short, *can* the Turing test ever succeed? Because what if a computer OVER-performs, giving away its true nature? It seems that for a Turing test to succeed, the computer must be on the exact cognitive level of a human being, not higher or lower.

Anyway... Hadoop is really cool and useful. Having another computer in the network would half processing times. Hadoop + custom programs + post-processing scripts sound like really useful ways to pre-process huge amounts of text before you let a machine learn days on end. Since the end product are files that are in a more "edible" format, it should be a lot faster to do the research cycle.

Confabulation theory

Confabulation theory is a theory from Robert Hecht-Nielsen about the cognitive function of the brain, or in other words the working of thought. It's a theory, not an explanation. Confabulation theory basically works by processing lots of information and then from this information, find out which symbols belong together. Those symbols that are often seen together (and in cases by which distance) is information contained within the network. Confabulation can now produce new sentences by continuously generating possible phrases based on the context that is seen prior to that.



In effect, the confabulation theory uses an architecture that can produce entirely new sentences, which are plausible in the context. The interesting thing is that these sentences are also grammatically and syntactically correct. Thus, the rules of a language seem to be embedded within this network.

This does not mean that the machine 'understands' the language, or that it is conscious of the sentences it produces. I think the results should be considered the production of a thought-less brain, which only has the capacity to produce sentences without understanding what they mean. It's probably comparable to the chinese room producer, where it looks at streams of chinese symbols. At some point, the machine understands the order in which the symbols may appear and which symbols are often seen together. When asked to produce a sentence on its own, it uses this knowledge to produce a sentence.

What interested me in the theory is how this network differs from other networks that can fantasize, like the RBM. The RBM is a network that stores knowledge by looking at things and can then complete the signal it is receiving. This confabulation network is slightly different, in the sense that it can project continuations (say, a hypothesis) that are very plausible. So, if you were building a network that can produce responses to sentences, the confabulation network is likely to perform better. But if you ask a confabulation network to recognize a face, it might have good difficulty and the RBM might be better.

The RBM is a flatter network (judging the entire system) in comparison to the confabulation network. The confabulation network just competes between symbols and modules and always takes the highest value of all, but since the signals proceed from those results towards other networks, it's in a sense hierarchical.

It'd be really interesting to be able to identify specific properties of each network and then see if they can be used together. It's also possible that we're thinking about this the wrong way. The continuous processing of the confabulation network is quite different from others. We like to think from static situation to the next, perhaps the entire thing is more dynamic than that and we should focus on generating states, looping back and reprocessing the results, thus continuously adding more results to some hypothesis.

Since A.I. is also a lot about search in search spaces (think chess!), a neural network could be used to generate a hypothesis step-by-step, until it is deemed that a particular branch isn't going to produce a good result, so that it can be terminated.

Philosophy of mind and innovation

In this post I'm going to talk about a possible measure of success for creating new innovations in software. To start off, I'll put some links to put this more into context: Here's a semi-graphical timeline of the development of the GUI. Here's one of my posts made in 2007, where I'm talking about how working with computers becomes more of a conversation than it has been before. Mainframes and the likes just accepted batch jobs and you'd type the command, hit enter and that's it. In web 2.0, the computer is more or less looking over your shoulder what you type, point at and do on the screen and then if it thinks it can help you out with something, it'll pop up some helpful hint or thing next to your focal point. It is related to what we know about and how we think about our bodies and minds.

And all of this started with philosophy, thus it started with the Greek: Aristotle, Plato and Socrates. The first thoughts there however were meta-physical. What is the world made of? What does it mean to be alive? In this context, the most important thought is the separation of body and soul. The years after that, most thinking is based on the philosophy of the greeks. The middle ages turned this around a bit with the introduction of religious thought into philosophy itself. Many people tried to explain things through the use of religious ideas. And then came the others to found modern philosophy, amongst them René Descartes.

Descartes is one of the founders of modern philosophy. In his discourse about the philosophy of mind, he introduced the concept of dualism. He also considered ideas about the working of the human body. The interesting thing here... the explanation used mostly analogies and was inspired by the progress of machinery, tools and things in real life that existed at the time. In that time, the functioning of nerves and blood vessels wasn't entirely clear. For 1,500 years, people thought that "animal spirits" governed the human body. However, during the time of Descartes, certain developments were underway, like the start of hydraulics. Basic, mechanic machines could be built. Descartes knew about those and imagined the body as a kind of automaton as well. To the best of his knowledge, he tried to explain how the body worked, and used the concepts that he had available to him. After Descartes, the thinking about psychology, the mind, the brain and how it all worked together really set off. In a sense, you could also say that the way how we started thinking about things just notched downwards one level.

At this time, the thoughts about the mind didn't really go further then: "there are mechanics at work that bring sensation to a central point somewhere in the brain". This central point was imagined to be a mystic piece of the puzzle [the soul?], where thoughts occur and what can be said to be the 'I, self', when you think. So, in that time, since people observing the machines in those days clearly understood that such a machine was not alive, they didn't (perhaps not all) imagine that the machine could actually produce human thought. But, some people that did not understand its function did say that the machine can be made to do anything we ask it to do.

The function and design of the machines were still used as analogies to explain how other processes could work in the human body. Physics and mechanics research has thus certainly helped the developments in medical science to better understand the function of the heart as a pump and the fluid dynamics of blood in the vessel.

This is just a tiny grasp from everything that's happened in thinking about the mind, surely, but there's not a lot of space left on this post to continue :). It's best to read it from this excellent source I found here: Courtesty Dr. C. George Boeree.

So, philosophy started with the Greek, seeding a new science called psychology along the way and in the 20th century already at the start, we started building computers.

Why is all this "philosophy stuff" relevant?

Well, if you look at the developments in philosophical thinking and the developments in machinery, they seem to go pretty well in step. This is not because they are directly tied together, but because other developments in neuroscience, psychology, electron microscopes and imaging techniques need to be developed in order to .... start asking the right questions. And philosophy, psychology, design, artificial intelligence and sciences are the most important scientific research sources for finding those questions and hopefully the answers.

Analogous to the comparison of the mind to the technology of every age, we also see that we (re)construct our ancient tools into newer technology available. But hold on there... Shouldn't this be new tools in new technology?

Consider the desktop for example. The GUI timeline at the top is a resemblance of how we introduced the computer to the general public. The GUI works great, I don't say it should never have been developed, but there are very old concepts still present in there. Take the desktop and its applications for example:

• The trash bin is literally there and even called the same.
  
• MS Powerpoint is basically a digital slideshow projector with edit functions.
• MS Word is clearly inspired by writing on paper, thus the typewriter.
  
• The "file manager" looks like a filing cabinet in the older GUI's, but this is slowly being replaced by "explorers". The explorer still has a hierarchical view of files however.

I think one of the reasons why this was done is to make it easier for users to familiarize themselves with the environment in which they were 'operating things'. But now that everyone is accustomed to the use of a computer, more or less, there doesn't seem to be a reason to maintain this ancient set of tools in this new environment.

So, one of the problems in software innovation is related to imagination, it's certainly not about technology. A good example for starting to get rid of powerpoint is prezi. Prezi is reminiscent of the concept of mindmapping, but that is great. What is the most important difference between prezi and powerpoint?

Powerpoint is a digitalization of the slideshow projector. Mindmapping is an attempt to convey thought and their relations between them. The first is putting an old thing into a new jacket with more features. The other is understanding the mind, understanding cognition, knowing your mind and how it works, how we consume information better, what allows us to make proper distinctions, what allows us to make better judgements, communication and what allows us to easily derive the correct context and then developing a method to enable that process.

So, to finish off my post in 2007: "Conversation with the machine" with the question:

"How can/will/should user-machine conversations evolve from this point onwards?"

I think innovation should focus on following and enabling the natural flow of cognitive processes, not on the reconstruction of ancient tools of communication and processing, like mail becoming e-mail. Before you feel the urge to send out a mail to someone, there is a reason, a motivation. What if we start from there instead and just consider the computer the ultimate tool in visualization and computation? Oh yeah, and it has connectivity as well.

( footnote: There are certainly other tools like keynote+iMovie and Adobe flash that can be used to produce a prezi-like presentation. Prezi is just mentioned, because I think it is a good example of how to think "out of the box" ).

SFTP transfer scripts

I've made a set of scripts to send files from one server to another in a secure and reliable way. The idea was to use the standard UNIX/Linux tools as much as possible, because this allows admins to customize the entire process to their hearts content. A new program using libraries would certainly be more limiting.

The project is published here:

http://code.google.com/p/sftp-transfer/

It is using SFTP to transfer files from A to B and the configuration is written with Ubuntu systems in mind. The entire setup can optionally use a chrooted sftp setup. The project demonstrates full design details and a complete installation manual.