Showing posts with label gwt. Show all posts
Showing posts with label gwt. Show all posts

Friday, August 28, 2009

Android: Piggy-backing on the WebKit browser

I've been looking for ways to not store or request username/password combinations for an Android app. The application I'm developing basically needs to get some data from a Google AppEngine application that is deployed, but that app uses authenticated users for doing some queries. I am actually reluctant to request username/password for the application to login, since I don't want people to even believe the combinations can be misused.

Ideally, I want the users to log in using some login provider on the android platform, or reuse login capabilities for gmail/calendar synchronization services, but I don't think that may be possible.

The objective is to just sync some data from the web, after the user logged in with their credentials. The login is then used to perform the query with. All this without needing to have another app register precious login details in its own space, which may get lost or recovered somehow by others.

Webkit to the rescue!

WebKit on Android can be used to show HTML help pages that were bundled along with the app. Also, you can programmatically construct a browser view then use events on the webkit browser to detect when a user finished its conversation (the login process) and then recover the cookies and use that to call the actual service with. The following snippest show how this can be used in practice:

WebView webview;

private class HelloWebViewClient extends WebViewClient {
@Override
public boolean shouldOverrideUrlLoading(WebView view, String url){
view.loadUrl(url);
return true;
}
@Override
public void onPageFinished(WebView view, String url) {
super.onPageFinished(view, url);
CookieManager mgr = CookieManager.getInstance();
Log.i( "HelloAndroid", url );
Log.i( "HelloAndroid", mgr.getCookie( url ) );
}
}

@Override
public void onCreate(Bundle icicle) {
super.onCreate(icicle);
setContentView(R.layout.main);

webview = (WebView) findViewById(R.id.webview);
webview.getSettings().setJavaScriptEnabled(true);
webview.setWebViewClient(new HelloWebViewClient());
webview.loadUrl("http://yourapp.appspot.com");
}

That's a good start for temporarily using the browser to do the login. The advantage is that username/password combinations never travel through your app and the standard web mechanism is used to obtain a set of cookies to use for navigating further. The username/password remain very volatile in this entire process.

Since I am sort of running the application in a browser, there are ways for this android app to 'step out' of the browser through one of the events and call local services. As a matter of fact, there is a very nice addJavascriptInterface call, which can register a specific interface in Javascript, which can be called by downloaded HTML pages. Certainly, care must be taken security-wise not to have any other pages loaded in the same browser instance that could also call the javascript interface.

class Logger
{
public void log(String content)
{
Log.i( "HelloAndroid", url );
}
}

..... setWebViewClient(new HelloWebViewClient());
webview.addJavascriptInterface(new Logger(), "Logger");
webview.loadUrl( ........

If then your remote HTML file contains:


window.Logger.log( 'hello Android!' );


You basically have a way to let your android phone do things through the remote server app. The WebViewClient should probably contain a couple of safety controls to disallow browsing to foreign destinations.

Why is this so interesting? because you can now create a server-based GWT app and create a nice-looking browser-based interface (generated or stored server-side) and possibly avoid a good deal of code for native interfaces.

It should even be possible to finish up the browsing part if the server dictates so and continue with some intent on the handset device (and possibly return later).

My intention is to just sync some information for another app and then use that data for other purposes locally. The browser allows me to use GWT all the way, so I can use that to do the very effective GWT-RPC calls to communicate data across effectively. The GWT client code can then call the pre-meditated interfaces in the android app, and we're done!

On towards the next phases... :)

Friday, July 17, 2009

Mapping demographic data, aggregated by geographic regions

Bit of a post for the Dutch, but you may have some data that is similarly detailed and cool (AND FREE!)... Together with a colleague I've been working on mapping demographic data of Holland on Google Maps for marketing purposes. The result is here:


There is a data dump from the CBS that you should be able to find with Google called "Districts and Neighborhoods" or "Wijken en Buurten". This data contains some demographic data about a region ( municipality, district and down to neighborhood ), as counted by the CBS. As well, there is a data set called "Core figures", which translated to "kerncijfers" also supplied by the CBS.

The core figures go down to the postal code level, which does two things:
  • Finer grained information than you already have (which is not very useful for marketing purposes)
  • Establish a correlation between postal code and the neighborhood it is in, thus allowing you to map groups of people to neighborhood level, which is probably a good aggregation level for marketing.
The district- and neighborhood data is delivered in the form of ESRI shape files. This allows you to convert the data into Well Known Binary format (WKB) using the shp2pgsql utility. This builds a table from the data in the SHP file and outputs this into sql into a file. Then you can simply < href="http://code.google.com/eclipse/docs/getting_started.html">Google plugin) + Hibernate. To be able to query the postgis database, you need the vivid solutions extensions.

Then I use a query to get the polygons out that I need through the postgis query API:

String wktFilter = getFilter( bounds );
WKTReader fromText = new WKTReader();
Geometry filter = null;
try{
filter = fromText.read(wktFilter.toString());
filter.setSRID( -1 );
} catch(ParseException e){
throw new RuntimeException("Not a WKT String:" + wktFilter);
}

Session s = HibernateUtil.currentSession();

Criteria testCriteria = s.createCriteria(Buurt.class);
testCriteria.add(SpatialRestrictions.intersects("theGeom",filter));
List buurten = testCriteria.list();

As such, I get the neighborhoods or other regions back that I want, based on my Google maps viewport :). That's already an important step, but doesn't finish there. Since the data in the Wijken en Buurten is RijksDriehoeksmeting (RD), it needs to be converted to the Datum that Google Maps uses. They're using WGS-84, a datum, which is basically a geoïd generally used for (probably) older GPS's. I'm using this code for now to convert between the two:

public static double[] rdtowgs( double X, double Y ) {
double dX = (X - 155000) * Math.pow( 10 , -5 );
double dY = (Y - 463000) * Math.pow( 10 , -5 );
double SomN = (3235.65389 * dY) + (-32.58297d * Math.pow( dX, 2)) + (-0.2475d * Math.pow( dY, 2)) + (-0.84978d * Math.pow( dX, 2) * dY) + (-0.0655d * Math.pow( dY, 3)) + (-0.01709d * Math.pow( dX, 2) * Math.pow( dY, 2)) + (-0.00738d * dX) + (0.0053d * Math.pow( dX, 4)) + (-0.00039d * Math.pow( dX, 2) * Math.pow( dY, 3)) + (0.00033d * Math.pow( dX, 4) * dY) + (-0.00012d * dX * dY);
double SomE = (5260.52916 * dX) + (105.94684d * dX * dY) + (2.45656d * dX * Math.pow( dY, 2)) + (-0.81885d * Math.pow( dX, 3)) + (0.05594d * dX * Math.pow( dY, 3)) + (-0.05607d * Math.pow( dX, 3) * dY) + (0.01199d * dY) + (-0.00256d * Math.pow( dX, 3) * Math.pow( dY, 2)) + (0.00128d * dX * Math.pow( dY, 4)) + (0.00022d * Math.pow( dY, 2)) + (-0.00022d * Math.pow( dX, 2)) + (0.00026d * Math.pow( dX, 5 ));
double ret[] = new double[ 2 ];
ret[ 0 ] = 52.15517d + (SomN / 3600);
ret[ 1 ] = 5.387206d + (SomE / 3600);
return ret;
}
public static double[] wgstord( double Latitude, double Longitude ) {
double dF = 0.36d * (Latitude - 52.15517440d);
double dL = 0.36d * (Longitude - 5.38720621d);

double SomX= (190094.945d * dL) + (-11832.228d * dF * dL) + (-144.221d * Math.pow( dF, 2 ) * dL) + (-32.391d * Math.pow( dL, 3) ) + (-0.705d * dF) + (-2.340d * Math.pow( dF, 3 ) * dL) + (-0.608d * dF * Math.pow( dL, 3 )) + (-0.008d * Math.pow( dL, 2 ) ) + (0.148d * Math.pow( dF, 2 ) * Math.pow( dL, 3 ) );
double SomY = (309056.544d * dF) + (3638.893d * Math.pow( dL, 2 )) + (73.077d * Math.pow( dF, 2 ) ) + (-157.984d * dF * Math.pow( dL, 2 )) + (59.788d * Math.pow( dF, 3 ) ) + (0.433d * dL) + (-6.439d * Math.pow( dF, 2 ) * Math.pow( dL, 2 )) + (-0.032d * dF * dL) + (0.092d * Math.pow( dL, 4 )) + (-0.054d * dF * Math.pow( dL, 4 ) );
double ret[] = new double[ 2 ];
ret[ 0 ] = 155000 + SomX;
ret[ 1 ] = 463000 + SomY;
return ret;
}


So there goes that! Any data coming back from the UI needs to be converted into RD first (the bounds), before querying the data on the viewport.

We're still working on refining things, like encoded polygons and possibly some caching, but in compiled form the app is a lot quicker than in the GWT shell. At the moment we're simply forwarding the individual points, but that's not as efficient and probably not as precise. That shouldn't take too long to get done however. Well, the rest of the application is just using the Google Map Library and putting that little map on the screen. That means using the proper events, hooks and scriptlets, to use the old MS word inbetween.

The figures are from 2004 and have been slightly updated for 2006, but probably not too much. The real figures about inhabitants are from 2009 and are based on the GBA, but some effort has been made to change things where problems could ensue regarding privacy.

What's the use? Well, together with the postal code, which is often requested, or with the customer's IP through geo targeting, you can start data mining. Marketers can quite easily develop certain profiles of what they're selling. By the IP the region can be quickly discovered, which may then give clues about preferences if the person is not known otherwise by login for example (any preferences or login that you have is far more useful than this silly method of customer targeting).

Based on the region/IP/postal code, you can find out what kind of products are more likely to suit the visitor. Thus, it provides a way to adjust the web content to the person that's visiting. Any other smaller clues like the first three clicks could theoretically tell you the rest of what the person is trying to do, or why your site is visited.

Some other sites like funda.nl use similar databases, although I believe they've probably paid around 7,000 EUR for a postal database which is slightly more precise. The CBS borders are those borders established by the government.

This little thing that you see on the page is interactive, only loads what it needs based on the viewport bounds and was hacked together in 16 hours.

Saturday, June 14, 2008

EK & GWT

Holland is playing really well in the EK. It's a joy to watch. Got 6 points in the pocket and all of Europe is commenting how well-greased the team is playing this year. I'm expecting Holland to win this EK.

Well, for something totally different, a new version of GWT is out, 1.5. The beta version has good new capabilities and standard themes that prove very useful. I'll be plugging that into my project and keep improving the user interface. Then probably release 1.5.1 before I'm hosting a demo next month at the company I work for, in front of project managers, architects, testers and other people.

Wednesday, July 04, 2007

Development with GWT, one year down the road

With GWT being relatively new, it is now approaching a final build and may very soon be upgraded to production status and taken out of Beta. This is a good time to reflect on one year of GWT usage and my experiences so far.

When GWT came out, I was on an overnight standby and had all the time to look into the technology. I looked at the samples and very quickly got very enthusiastic. Now after one year, I still see the immense value that the toolkit provides and since the first release, many very important and cool changes and additions were made.

When Java came out, there was a lot of hype around the technology. Nowadays the Java language is accompanied by good practices and some people even have written rather large books about Java Design and coding patterns.

There are things one should know for GWT development as well. Many of the regular J2EE patterns are not exactly applicable to GWT overall, so many of the historic J2EE patterns are simply not useful.

A couple of things that I think every project should think out before starting on a GWT project:
  • What is the final size of the project? Will it be possible to fit this in a single module (compiler memory usage) or is it necessary to modularize from the start? If so, how will this be modularized?
  • How to structure components, modules and libraries to facilitate re-use of development in other projects as well? Use of imports!
  • It is highly recommended to think about a strategy for history browsing and perma-links. See History class of GWT and "onHistoryChanged".
  • Focus on the re-use of widgets and make developers aware of the importance of abstraction and reuse. I have found it is much more important to develop components that can be reused in different contexts than it is to solve a particular problem at hand in a particular way. Make sure to review that code.
  • Test the application on different platforms and on different browsers as you go along.
  • Develop the application on different platforms too. It makes sense to use Linux with FireFox and Mozilla by 2-3 developers of the project, where the other half uses Windows with maybe different versions of IE.
  • If the project will get very large, consider running in "noserver" mode from the start of the project. You will have to model your development environment slightly differently to be able to achieve this.
  • Develop proto-types of screens from the start. Do not develop screens before the proto-types are ready and you have an idea of the final Look and Feel of the application overall.
  • Hire a CSS expert. Make sure that your CSS tag names and approaches are consistent, make sense with your developers and its development is aligned with the HTML approach that GWT embeds. Code the general look and feel into the "gwt-" tags. For cases that require different approaches, derive from other tags and apply the differences there.
  • Give back to the community those cool things you are developing or document your innovative approaches. Post on the GWT newsgroups or help out with the development of one of those UI libraries out there. Some companies require specific legal sign-offs for contributing to open source projects.
  • Program against interfaces where applicable, not against specific classes.
  • Usability becomes much more important on the Internet. It's not just putting together some HTML pages anymore. Make sure you have someone on the team that understands usability issues and can design useful, easy screens for people to use.
Hope that helps. Good luck in your GWT projects!

Thursday, June 21, 2007

GWT 1.4, Tomcat 6 and Comet tutorial

I've been experimenting with Comet applications a little bit and integrated this with GWT 1.4. The tutorial is sort of working, but still needs a bit of work in order to become more stable.

The tutorial is here:

http://gtoonstra.googlepages.com/cometwithgwtandtomcat

Please send me any comments, bug fixes, etc..

Friday, March 23, 2007

Guice in GWT

I am separating some functionalities in Project Dune. Transaction control was initially handcoded in each service request, but there were two problems here. The service request started the transaction in an HTTP specific stub and the business logic was mixed with this protocol-specific code. So the separation puts the business code into a separate object and transaction control is managed on any method that this business object does.

The first attempt actually was a transaction filter that always creates a transaction and then closes it, but this is expensive and one problem with GWT is that the SerializableException gets consumed and serialized as a response stream. So the filter will never see this exception being raised.

I have thus used Guice (pronounce this as "juice") to deal with these problems. The way it works is like this:
  1. A servlet context listener is used (see web.xml configuration) to initialize a Guice Injector and bind this to an attribute into the servlet context.
  2. A servlet filter always opens a Hibernate session. (not a transaction!). It will also (try-finally) close this session.
  3. When a GWT service needs to service a request, Tomcat will create the specified servlet.
  4. Each servlet derives from a BaseServiceImpl abstract servlet.
  5. The BaseServiceImpl overrides "init( ServletConfig config )". Through the "config.getServletContext().getAttribute()"
  6. I am retrieving the injector object, created in the listener.
  7. The injector object calls "injector.injectMembers( this )", which will "instrument" any annotated members in the servlet instance.
  8. When the injector sees a request to inject a field or method parameter, it will look this up in the registry and also attempt to inject any annotations that may exist in the to-be-injected instance.
  9. And so through very simple annotations, it may result in a couple of cascaded injection requests when a servlet gets instantiated.
The very nice thing about Guice is that you no longer have to deal with XML files. It is all programmatic. As soon as you have your "injector" instance, this instance will have been configured with a couple of bindings. Those bindings have matchers on classes and methods and if it finds anything that is annotated, it will perform that instrumentation.

Notice however that Guice is *not* installed on the classloader. This means that just setting "@Inject" on a field for example will not do anything *unless* you retrieve the instance through the injector instance. This latter part is not very easy for
everybody to understand right away, but is the most important aspect (no pun intended) about Guice programming as I have found so far.

Code example? You will need aopalliance.jar and guice-1.0.jar for this to run, downloadable from the Guice website:

ServletContextListener:
================
public class GuiceServletContextListener implements
    ServletContextListener
{
    public GuiceServletContextListener() {
        super();
    }

    public void contextInitialized(ServletContextEvent servletContextEvent)
    {
        ServletContext servletContext =
            servletContextEvent.getServletContext();

        // Create our injector for our application use
        // store it in servlet context.
        Injector injector = Guice.createInjector( new TransactionModel() );
        servletContext.setAttribute( Injector.class.getName(), injector );
    }

    public void contextDestroyed(
        ServletContextEvent servletContextEvent)
    {
    }
}

TransactionModel:
=============
public class TransactionModel implements Module
{
    public void configure(Binder binder)
    {
        binder.bindInterceptor(
            any(), // Match classes.
            annotatedWith(Transactional.class), // Match methods.
            new TransactionInterceptor() // The interceptor.
        );
    }
}

Transactional:
==========
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD})
public @interface Transactional {
}

BaseServiceImpl (a base class for any servlet in the application):
==========================================

public abstract class BaseServiceImpl extends RemoteServiceServlet {
    .....
    @Override
    public void init(ServletConfig config) throws ServletException {
        super.init(config);
        Injector injector = (Injector)config.getServletContext().
            getAttribute( Injector.class.getName() );
        injector.injectMembers( this );
    }
    .....
}

CustomerServiceImpl (implementation of GWT service):
====================================
public class CustomerServiceImpl extends BaseServiceImpl implements CustomerService {
    ......
    @Inject
    private CustomerBO customerBO;
    ......
    public CustomerDTO saveCustomer( CustomerDTO dto, boolean isNew )
        throws UIException
    {
        try {
            Customer customer = customerBO.getCustomer( dto.getCustomerId() );
            if ( customer == null ) {
                checkAccess( WebConstants.CUSTOMER_ADD );
                // not found, so create it.
                customer = new Customer();
            } else {
                checkAccess( WebConstants.CUSTOMER_EDIT );
            }

            MapperIF mapper = DozerBeanMapperSingletonWrapper.getInstance();
            mapper.map( custDTO, customer );

            customer = customerBO.saveCustomer( customer, isNew );
            CustomerDTO dto = customerBO.getCustomer( customer.getCustomerId() );

            return dto;
        } catch (ApplicationException ae ) {
            log.error( "Could not save customer", ae );
            throw new UIException( ae.getMessage() );
        }
    }

    ......
}

CustomerBO:
=========

@Singleton
public class CustomerBO extends BaseBO {
    ......
    @Transactional
    public Customer saveCustomer( Customer customer, boolean isNew )
        throws ApplicationException
    {
        Session session = getSession();

        try {
            if ( isNew && customer != null ) {
                throw new ApplicationException(
                    custI18n.getMessage( "error.cust.already.exists" ) );
            }

            // validate will raise an ApplicationException if the customer data is invalid.
            validateCustomer( customer );

            if ( isNew ) {
                session.save( customer );
            } else {
                session.update( customer );
            }

            customer = getCustomer( customer.getCustomerId() );

            return customer;
        } catch (HibernateException he ) {
            log.error( "Could not save customer", he );
            throw new ApplicationException(
                custI18n.getMessage( "error.save.customer" ));
        }
    }
}

==========================

Obviously this code can/should be extended with a variety of things. It should probably check if there is already a transaction ongoing. It should probably add parameters to the transaction interface to find out how the method supports transactions (required, supports, requiresNew, etc) and so on. But for simplicity's
sake, this is the bare minimum.

Notice how, once you have started the injector in the ServiceImpl, the CustomerBO does not need to be declared specifically in the injector. This is some sort of automatic cascading injection effect which happens because the injector is already processing dependent classes. So, luckily, you only need to use the Injector once, which will inject all your other classes where you want them.

Also have a look how to do this for interfaces. What is lacking in the CustomerBO is a separation of persistence with business logic. If you separate this further, you have a start to be able to switch the implementation of your persistence framework.

I am personally contemplating to bring the transaction boundary more forward and wrap this around the methods (where required) of ServiceImpl instead. But I am not sure whether this will work.

Good luck on your own incantations!