tag:blogger.com,1999:blog-30594688132558680462024-02-02T17:41:25.511+00:00Swish MovementGeeky ramblings of a Java Server Side developer.Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.comBlogger24125tag:blogger.com,1999:blog-3059468813255868046.post-77333522342303035522013-01-17T13:43:00.001+00:002013-01-17T13:44:47.758+00:00Distributed transactions with Spring<b>Warning:</b> The application design I describe below is not one that I'd recommend. This post concerns slight improvements to a monster of a legacy project that has some technical debt. Specifically I'm not at all happy with the following:<br />
<ul>
<li>It uses a mix of ORM technologies</li>
<li>It is multi-VM where it should be multi-threaded</li>
<li>The current design requires distributed transactions whereas a better design might not</li>
</ul>
End of warning.<br />
<br />
I’ve been working on some improvements for a legacy data loading application. It basically takes in a bunch of XML in different formats and updates a couple of separate databases with the transformed data. One database maintains a history of the loaded data, the other a current snapshot. It is essential that the entities referenced in these databases are consistent with one another.<br />
<br />
Now, given that there are two databases that are both being updated and that some keys in database A are referenced in database B, I was expecting to see some kind of distributed transaction control. However, some sample test cases showed that this was not the case - a rollback on one database would not initiate a rollback on the other - leaving the databases in an inconsistent state.<br />
<br />
A simple solution would be to use a <a href="http://en.wikipedia.org/wiki/Java_Transaction_API">JTA</a>/XA transaction manager to provide a robust distributed transactions with <a href="http://en.wikipedia.org/wiki/2PC">2 phase commits</a>. I’ve use <a href="http://en.wikipedia.org/wiki/X/Open_XA">XA</a> in the distant past where the transaction management is provided by a J2EE container such as WebLogic or JBoss, I seem to recall it being fairly straightforward - pretty much checking an ‘XA’ checkbox on each data source. However, the application in question has no such container and uses Spring instead - I was not keen to introduce a container just to fix this problem.<br />
<br />
Fortunately there are a couple of open source JTA providers that are supposed to play nicely with Spring - the most visible being <a href="http://www.atomikos.com/Main/TransactionsEssentials">Atomikos TransactionEssentials</a> and <a href="http://docs.codehaus.org/display/BTM/Home">Bitronix BTM</a>. These solutions are also compatible with Hibernate and iBatis - another requirement as <b>both</b> of these technologies are used.<br />
<h4>
Atomikos TransactionEssentials</h4>
TransactionEssentials seemed to be the most popular open source JTA provider, so I gave this a go first. Integrating it was relatively straightforward:<br />
<ul>
<li>Create a couple of Atomikos beans and a property file</li>
<li>Switch from the HibernateTransactionManager to the JtaTransationManager</li>
<li>Set the data source implementations to the respective XA proxy provided by Atomikos</li>
<li>Configure the database driver class to be an XA variant (<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">org.postgresql.xa.PGXADataSource</span> in my case)</li>
<li>Provide some additional configuration to the Hibernate session factory.</li>
</ul>
With only these Spring configuration changes I was able to get XA transactions working. The test cases that had previously failed now passed. However, I ran into some annoying problems with our integration test suite. It looked as though the Atomikos datasources registered in one test class were still present when running other test classes. This caused a frequent ‘<i>Another resource already exists with name</i>’ error from Atomikos. Initially I suspected this could be a context caching issue with the <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">SpringJUnit4ClassRunner</span>. However, after carefully applying <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">@DirtiesContext</span> directives to the relevant tests, and changing the Maven SureFire fork mode to ‘always’ the problem persisted. After spending some time trying to get the tests to cooperate I decided to try another JTA provider as I suspected that Atomikos was responsible for the persistence of the data source configuration between tests and that ultimately this was the root of my problem. Also, given that the integration of Atomikos was relatively smooth - I expected that a transition to another JTA provider would be even easier.<br />
<h4>
Bitronix BTM</h4>
Bitronix also seemed fairly popular and looked to be reasonably actively maintained so I chose to try this out next. Integration was trivial this time around:<br />
<ul>
<li>Swap a set of Atomikos Spring beans with a smaller number of Bitronix beans</li>
<li>Change XA driver classes from Atomikos versions to the Bitronix equivalents</li>
<li>Modify a few driver property names</li>
</ul>
For those who are interested - I’ve made my Spring/Hibernate/Bitronix <a href="https://gist.github.com/4555886">configuration available</a>. When I fired up my test suite this time around everything worked perfectly so the use of Bitronix BTM seemed to give a quick win - however, there were some problems were waiting for me when deploying and running the full application.<br />
<h4>
Avoiding JNDI</h4>
<div>
Hibernate's default <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">JTATransactionFactory</span> implementation uses JNDI to lookup the <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">UserTransaction</span>. This might make sense for applications running in a container, but we have no need for JNDI in our Spring based configuration. Ultimately I can obtain the <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">UserTransaction</span> instance directly from the application context. To wire this into Hibernate I created a <a href="https://gist.github.com/4555896">simple <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">JTATransactionFactory</span></a> that bypasses JNDI with a direct reference to the bean. This has to be instantiated and configured in the context and must be created before the <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">SessionFactory</span>.</div>
<h4>
Multiple processes using Bitronix on a single machine</h4>
Unfortunately, the legacy application in questions runs as multiple separate JVM processes on a single machine. The application really should have been implemented in a multi-threaded way but unfortunately for me - the current maintainer - it wasn’t and a big refactor is not feasible at this time. This multi-process pattern posed a problem for Bitronix as it has some default locations where it stores transaction logs and we don’t want more than one transaction manager updating a given log file. Fortunately, Bitronix can use alternative log name configurations and I was able to set a unique log file name per process:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">bitronix.tm.journal.disk.logPart1Filename=/var/run/myapp/${nodeName}-btm1.tlog</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">bitronix.tm.journal.disk.logPart2Filename=/var/run/myapp/${nodeName}-btm2.tlog</span><br />
<h4>
Additional database resources used by XA</h4>
The databases in question were both running in PostgreSQL instances. For the vendor XA driver to function the database server's <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">max_prepared_transactions</span> configuration option must be set to a suitable non-zero value (it is 0 by default). This configuration option only takes effect after a restart. In my case, for one of the database servers I’m using a restart is no small matter - so it just wasn’t practical to change this setting in the short-term. However, I still needed to get my fix out to improve the transactional behaviour of the application as soon as possible.<br />
<br />
I decided to forego a little robustness and use a <a href="http://docs.codehaus.org/display/BTM/LastResourceCommit">Last Resource Commit optimization</a>. This allows for one non-XA resource to participate in the distributed transaction. The transaction manager always commits transactions on this datasource last so that it can still rollback the XA datasources if the commit fails. Implementing the optimization was straightforward - I switched the XA datasource proxy implementation for the database in question to <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">bitronix.tm.resource.jdbc.lrc.LrcXADataSource</span>. With this in place I had everything working again.<br />
<h4>
Summary</h4>
Note that although the integration of JTA/XA can be fairly straightforward, it is essential that you have test cases to verify the expected transactional behaviour of your application. Do not assume that it’ll just work.Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com0tag:blogger.com,1999:blog-3059468813255868046.post-77350214821644342382012-10-13T09:00:00.000+01:002012-10-13T13:15:03.840+01:00Last.fm open source commons librariesWe recently open-sourced a few our our common libraries at <a href="http://last.fm/">Last.fm</a>. This was primarily because we had a number of larger projects that we wanted to make public such as <a href="https://github.com/lastfm/moji">Moji</a> - our Java <a href="http://code.google.com/p/mogilefs/">MogileFS</a> client. These projects have a dependencies on convenience classes in our common libraries. Rather than duplicate these classes we decided to also make the effort to open source the libraries themselves.<br />
<br />
These libraries contain a few classes that I find particularly useful when writing tests:<br />
<h4>
<a href="https://github.com/lastfm/lastcommons-lang">lastcommons-lang</a></h4>
This library contains a nice system clock abstraction named (unsurprisingly) '<a href="https://github.com/lastfm/lastcommons-lang/blob/master/src/main/java/fm/last/commons/lang/time/Clock.java">Clock</a>' which was inspired by this <a href="http://stackoverflow.com/a/2001696/74772">StackOverflow answer</a>. A clock instance allows you to get access to various forms of the current time. So far, millis, nanos, <span style="font-family: Courier New, Courier, monospace;">java.util.Date</span>, <span style="font-family: Courier New, Courier, monospace;">java.util.Calendar</span>, and <span style="font-family: Courier New, Courier, monospace;">org.joda.time.DateTime</span> instances are supported. The class does seem too simple to be of any value - after all it just delegates to some standard Java API methods. However, the real power of the abstraction comes when unit testing time dependent code that you may have.<br />
<br />
In your unit tests you now have the ability to swap out the actual system clock instance with a mock or a <a href="https://github.com/lastfm/lastcommons-test/blob/master/src/main/java/fm/last/commons/test/time/StubClock.java">stub</a> and take control of the 'current time' - at least as far as the class under test is concerned. This can be very useful as time dependent tests that use the system clock directly can often suffer from the following problems:<br />
<br />
<b>Long running tests</b><br />
You may have some code that should change some state once a minute has elapsed. If using the system clock directly you might call <span style="font-family: Courier New, Courier, monospace;">Thread.sleep()</span> to let allow the system clock to advance:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> myClass.initialise();</span><br />
<span style="font-family: Courier New, Courier, monospace;"> Thread.sleep(TimeUnit.MINUTES.toMillis(1));</span><br />
<span style="font-family: Courier New, Courier, monospace;"> myClass.changeState();</span><br />
<span style="font-family: Courier New, Courier, monospace;"> assertThat(myClass.getState(), is(/* changed state */));</span><br />
<br />
This will work, but will also result in a test that take no less than one minute to execute. With more tests such delays are compounded and we might end up with a test suite that takes some considerable time to execute. We want tests to be fast so that they are unnoticeable and easy to run. With a mockable clock we could instead write the following and be free from long delays:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> when(mockClock.currentTimeMillis()).thenReturn(</span><br />
<span style="font-family: Courier New, Courier, monospace;"> 0L, </span><span style="font-family: 'Courier New', Courier, monospace;">TimeUnit.MINUTES.toMillis(1)</span><span style="font-family: Courier New, Courier, monospace;">);</span><br />
<span style="font-family: Courier New, Courier, monospace;"> myClass.setClock(mockClock);</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> myClass.initialise();</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> myClass.changeState();</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> assertThat(myClass.getState(), is(/* changed state */));</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><b>Delays in tests are brittle</b></span><br />
You might have another test on a class that should only change state if more than 5 seconds but less than 10 seconds has passed. The problem here is that <span style="font-family: Courier New, Courier, monospace;">Thread.sleep()</span> does not guarantee the exact duration of the delay. If your machine is busy the system clock might easily tick by 5 seconds before your thread on which your test is running is rescheduled and your assertion will fail. I have seen this frequently with tests running on continuous-integration machines - tests may run fine for months but if the CI machine is particularly busy, brittle time dependent tests will suddenly start failing. As shown in the previous example - a clock abstraction can removed the need for such delays.<br />
<br />
<b>The future</b><br />
You may wish to test the class for future moments in time and unless you have a time machine - this just isn't practical with the system clock alone. However, you can set a mock or stub clock to whatever time you wish.<br />
<h4>
<a href="https://github.com/lastfm/lastcommons-test">lastcommons-test</a></h4>
We really like using JUnit's <span style="font-family: Courier New, Courier, monospace;"><a href="http://weblogs.java.net/blog/johnsmart/archive/2009/09/29/working-temporary-files-junit-47">TemporaryFolder</a></span> rule for cleanly managing temporary folder and files in our tests. I automatically handles the tear down of the temp fails and removes the need for unsightly path strings in the test code, which in the worst case often end up in a shared constant somewhere. We applied a similar approach to the handling of data files for tests using a <a href="https://github.com/lastfm/lastcommons-test/blob/master/src/main/java/fm/last/commons/test/file/DataFolder.java"><span style="font-family: Courier New, Courier, monospace;">DataFolder</span></a> abstraction. Assuming our standard Maven project layout, implementations allow simple, clean and scoped access to test data. The data folders also allow us to organise test data by test class or even test method without having to worry about the underlying file path. Some examples:<br />
<h4>
Top level data folder</h4>
<span style="font-family: Courier New, Courier, monospace;"> public class MyTest {</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> @Rule</span><br />
<span style="font-family: Courier New, Courier, monospace;"> public DataFolder dataFolder = new <a href="https://github.com/lastfm/lastcommons-test/blob/master/src/main/java/fm/last/commons/test/file/RootDataFolder.java">RootDataFolder</a>();</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> @Test</span><br />
<span style="font-family: Courier New, Courier, monospace;"> public void myTestMethod() throws IOException {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> File actualFolder = dataFolder.getFolder();</span><br />
<span style="font-family: Courier New, Courier, monospace;"> // Path: ./src/test/data</span><br />
<span style="font-family: Courier New, Courier, monospace;"> ...</span><br />
<h4>
Top level data folder with children</h4>
<span style="font-family: Courier New, Courier, monospace;"> public class MyMp3Test {</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> @Rule</span><br />
<span style="font-family: Courier New, Courier, monospace;"> public DataFolder dataFolder</span><br />
<span style="font-family: Courier New, Courier, monospace;"> = new <a href="https://github.com/lastfm/lastcommons-test/blob/master/src/main/java/fm/last/commons/test/file/RootDataFolder.java">RootDataFolder</a>("mp3", "128K", "clean");</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> @Test</span><br />
<span style="font-family: Courier New, Courier, monospace;"> public void myTestMethod() throws IOException {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> File actualFolder = dataFolder.getFolder();</span><br />
<span style="font-family: Courier New, Courier, monospace;"> // Path: ./src/test/data/mp3/128k/clean</span><br />
<span style="font-family: Courier New, Courier, monospace;"> ...</span><br />
<h4>
Per-class data folder</h4>
<span style="font-family: Courier New, Courier, monospace;"> public class MyTest {</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> @Rule</span><br />
<span style="font-family: Courier New, Courier, monospace;"> public DataFolder dataFolder = new <a href="https://github.com/lastfm/lastcommons-test/blob/master/src/main/java/fm/last/commons/test/file/ClassDataFolder.java">ClassDataFolder</a>();</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> @Test</span><br />
<span style="font-family: Courier New, Courier, monospace;"> public void myTestMethod() throws IOException {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> File actualFolder = dataFolder.getFolder();</span><br />
<span style="font-family: Courier New, Courier, monospace;"> // Path: ./src/test/data/fm/last/project/MyTest</span><br />
<span style="font-family: Courier New, Courier, monospace;"> ...</span><br />
<h4>
Per-method data folder</h4>
<span style="font-family: Courier New, Courier, monospace;"> public class MyTest {</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> @Rule</span><br />
<span style="font-family: Courier New, Courier, monospace;"> public DataFolder dataFolder = new <a href="https://github.com/lastfm/lastcommons-test/blob/master/src/main/java/fm/last/commons/test/file/MethodDataFolder.java">MethodDataFolder</a>();</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> @Test</span><br />
<span style="font-family: Courier New, Courier, monospace;"> public void myTestMethod() throws IOException {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> File actualFolder = dataFolder.getFolder();</span><br />
<span style="font-family: Courier New, Courier, monospace;"> // Path: ./src/test/data/fm/last/project/MyTest/myTestMethod</span><br />
<h4>
Wrap up</h4>
Hopefully you may find these libraries or the concepts presented here useful in your own tests.Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com0tag:blogger.com,1999:blog-3059468813255868046.post-52225546078896756542012-10-12T20:00:00.000+01:002012-10-13T13:14:49.688+01:00Garbage Collector tuning for services with large heapsI've been working on a <a href="http://thrift.apache.org/">Thrift</a> based Java service that uses a large heap. Most of the heap is occupied with a fairly static 7GB dataset that is reloaded every 24 hours. Real-time updates to the data occur throughout the day but they are low in both volume and size, and are incorporated into the next days 'full' dataset. The rest of the heap is made up of very short-lived objects necessary to represent and process client requests to the service. Ideally the service should have a response latency in the order of milliseconds and it met this requirement easily on its initial deployment using just the Concurrent-Mark-Sweep garbage collector.<br />
<br />
More recently the traffic has increased by an order of magnitude. In these conditions the default garbage collector configuration couldn't cope and 'stop the world' pauses lasting many seconds were killing performance. We were analysing the garbage collector operation with the following JVM options:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> -verbose:gc</span><br />
<span style="font-family: Courier New, Courier, monospace;"> -Xloggc:/var/log/service-name/gc.log</span><br />
<span style="font-family: Courier New, Courier, monospace;"> -XX:+PrintGCDetails</span><br />
<span style="font-family: Courier New, Courier, monospace;"> -XX:+PrintGCTimeStamps</span><br />
<br />
We found that the garbage collector was collecting across the large old generation. However, in the expected usage scenario the old generation should ideally contain only the long lived static dataset, perhaps with the real-time data updates that are delivered throughout the day - nothing should need to be collected from it. The GC logs showed that many more objects were frequently tenured into the old generation only to be later garbage collected. So where were all of these objects coming from? The most likely source were the groups of 'short-lived' objects created per request - if these were being promoted into the old generation then it could grow quickly, requiring frequent long collections. So how can you prevent this from happening?:<br />
<ol>
<li>Increase the maximum tenuring age to increase the window of opportunity for young objects to be collected before they reach the old generation.</li>
<li>Increase the size of the new generation to lengthen the interval between new space collection cycles which in turn can make it less likely that young objects will be promoted to the old generation.</li>
</ol>
<h4>
Increasing the maximum tenure threshold</h4>
On each young generation GC cycle, the garbage collector promotes surviving objects from the eden space into a survivor space. There are two survivor spaces and CMS alternates between them on each cycle. There is also a tenure counter on each object in the survivor space that notes the number of GC cycles the object has been there. Every time the survivor space is collected, all objects that remain in the space have their counter incremented. Once an objects counter reaches the maximum tenure age, it is promoted to the old generation.<br />
<br />
By keeping objects in the smaller survivor spaces for longer we can hope that objects will have to stick around for a significant amount of time before being promoted to the old generation. In this time we can hope that any truly short-lived objects will be collected long before they hit the maximum tenure count. The default maximum tenure threshold is 4, but it can be configured as high as 15 like so:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> -XX:MaxTenuringThreshold=15</span><br />
<h4>
Increasing the size of the new generation</h4>
<div>
Ideally we want all short-lived objects to be collected early on from the new generation. However, if the new space is small then there will be many frequent small collections. When large object throughput is concerned, eden space will fill up quickly - and if the requests are still in progress many objects will enter the survivor spaces as they are still 'in use'. As the collections are frequent, the tenure counters on the objects in the survivor spaces will increment quickly. Should these objects still be 'in use' by the time they reach the maximum tenure age then they'll enter the old generation - something that we want to avoid. To slow this process down we can increase the size of the new space like so:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> -Xmn3g</span></div>
<div>
<br /></div>
<div>
When increasing the new space we were actually more concerned with increasing eden space. If the time it takes to fill eden (and thus cause a collection) is significantly greater than the average request duration then we could hope that the majority of the newly created objects will be 'dead' by the time a collection starts and are hence immediately eligible for collection from eden. Thanks to the increase in tenuring threshold, those that do survive eden will then have to make it through 15 more GC cycles in the survivor space before they are promoted to the old generation. A arbitrary new allocation of 3GB was chosen as a good starting point.</div>
<h4>
Monitoring the garbage collector</h4>
With these settings in place we wanted to get a good grasp of how they affected the garbage collection profile of my application. Firstly we enabled some extra GC logging to provide insights on the tenuring of objects:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> -XX:+PrintTenuringDistribution</span><br />
<br />
We also started an instance of <a href="http://visualvm.java.net/">VisualVM</a> on the same machine and installed the <a href="https://blogs.oracle.com/nbprofiler/entry/visual_gc_plugin_2_0">VisualGC</a> plugin. This plugin provides an excellent insight to what's going on inside the garbage collector in real-time and can really aid understanding - especially when compared with the rather obtuse GC log formats.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhM2ynFF89aljkKJKR5GTM__Ujpx8vd4iLTibAJig_zB7I8ezmJ1chG5vqKks2bOimOTy9JB9FQXAY6d_m37GmD2oX0M8rVZWgnU7abPcaQu4AlwV5mIqS5IpwSTXdFqWaiEYL0jk2b19ys/s1600/visualgc.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhM2ynFF89aljkKJKR5GTM__Ujpx8vd4iLTibAJig_zB7I8ezmJ1chG5vqKks2bOimOTy9JB9FQXAY6d_m37GmD2oX0M8rVZWgnU7abPcaQu4AlwV5mIqS5IpwSTXdFqWaiEYL0jk2b19ys/s320/visualgc.png" width="303" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">VisualGC plugin for VisualVM</td></tr>
</tbody></table>
The VisualGC above shows a snapshot of the GC as the service is load tested. The 7GB dataset is the big green block labelled 'old'. The new collections (dark brown saw-tooth pattern labelled 'eden') are regular as the load is fairly constant, and thus so is the object throughput. It actually takes quite a while for eden to fill up so we could likely decrease the new size from 3GB with out any ill effect. The fact that the survivor spaces (S0, S1, and Histogram) are so empty shows that nearly all per-request objects are disposed of from eden and never make it to the survivor spaces. The 'histogram' section shows that the objects that do make it into survivor fall away rapidly with each cycle so that very little is tenured into the old. This is great because the old space is now effectively static and the GC won't have any cause to try and perform a large collection on it. However, we were curious to see which objects <b>were</b> making it into survivor, so for that we turned to the tenuring distributions in the GC logs (sample edited for brevity):<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> 2293.760: [GC 2293.760: [ParNew</span><br />
<span style="font-family: Courier New, Courier, monospace;"> Desired survivor size 161054720 bytes, </span><span style="font-family: 'Courier New', Courier, monospace;">new </span><span style="font-family: 'Courier New', Courier, monospace;">threshold 15 (max 15)</span><br />
<span style="font-family: Courier New, Courier, monospace;"> - age 1: 8344528 bytes, 8344528 total</span><br />
<span style="font-family: Courier New, Courier, monospace;"> - age 2: 2035920 bytes, 10380448 total</span><br />
<span style="font-family: Courier New, Courier, monospace;"> - age 3: 4351424 bytes, 14731872 total</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> - age 4: 4351424 bytes, 14731872 total</span><br />
<span style="font-family: Courier New, Courier, monospace;"> ...</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"> - age 15: 1165424 bytes, ...</span><br />
<div>
<br /></div>
<div>
What this log entry shows is that around 10KB of data is making it through to maximum tenuring age. Certainly expected some objects to occasionally reach this point - such as those representing updates to the dataset - however, looking across multiple log entries it was evident that the rate of object promotion to the old space was consistent - perhaps something was leaking? It would be extremely convenient if there was some way of asking the GC <i>'which objects do you have in tenure bucket 15?'</i> because then we'd then know exactly what is being promoted - unfortunately there is no way of doing so.</div>
<div>
<br /></div>
<div>
Luckily we did have was a 'good guess'. The service makes use of a monitoring component that uses data buckets to aggregate application metrics over time - each bucket typically holds a minutes worth of data and is valid for up to ten minutes. It seemed likely that these buckets would be alive long enough to be promoted to old and - in the worst case - fill up the old space. We decided to rewrite this component to use fixed-length queues that are allocated one on construction. When testing the service with this change the GC logs changed dramatically:<br />
<br />
<span style="font-family: 'Courier New', Courier, monospace;"> 5755.168: [GC 5755.168: [ParNew</span><br />
<span style="font-family: Courier New, Courier, monospace;"> Desired survivor size 161054720 bytes, new threshold 15 (max 15)</span><br />
<span style="font-family: Courier New, Courier, monospace;"> - age 1: 9638328 bytes, 9638328 total</span><br />
<span style="font-family: Courier New, Courier, monospace;"> - age 2: 79056 bytes, 9717384 total</span><br />
<span style="font-family: Courier New, Courier, monospace;"> - age 3: 24 bytes, 9717408 total</span><br />
<span style="font-family: Courier New, Courier, monospace;"> - age 4: 56 bytes, 9717464 total</span><br />
<span style="font-family: Courier New, Courier, monospace;"> ...</span><br />
<span style="font-family: Courier New, Courier, monospace;"> - age 14: 1024 bytes, 9724088 total</span><br />
<span style="font-family: Courier New, Courier, monospace;"> - age 15: 32 bytes, ...</span><br />
<div>
<br /></div>
This log shows an extremely low movement of objects from the new space to the old.<br />
<br />
So, we were able to use VisualGC and GC debug logs to provide insight to the garbage collection process. This helped us tune our application so that long pauses were effectively eradicated and latency targets could be reliably met. Of course, a full GC will happen eventually, but our architecture conveniently allows for this to be on our terms (when running under normal conditions). Different architectures may require alternative garbage collection strategies, but I recommend profiling with VisualGC early on and then using the GC logs to provide greater detail.</div>
<div>
<br /></div>
<div>
I must offer my thanks to: my colleague <a href="https://github.com/Noddy76">James</a> for pairing up with me on this problem!</div>
<div>
<br /></div>
<br />
<br />
<!--EndFragment-->
Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com0tag:blogger.com,1999:blog-3059468813255868046.post-26171457551922317902012-10-08T14:58:00.000+01:002012-10-08T15:05:25.766+01:00Setting ulimits for jsvcOne of my <a href="http://thrift.apache.org/">Thrift</a> based Java services was recently rolled out to a new set of clients, resulting in a 2000% increase in requests. This was all planned for and the transition was 'fairly' smooth. However, I noticed that the service would run out file descriptors periodically. This would result in the following exception:<br />
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> java.io.IOException: Too many open files</span></div>
<div>
<br /></div>
<div>
Now when we had planned to deploy the service we had considered the number of concurrent connections and set the open file limit for the user under which the service would run. So, the presence of this error was quite confusing. I initially suspected a file leak so started graphing the number of files the process had open using a simple script to capture the data - note that as I'm using <a href="http://commons.apache.org/daemon/jsvc.html">jsvc</a>, it's fairly easy to identify the process ID:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> #!/bin/bash</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> PID=`cat /var/run/service-name/service-name.pid`</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> echo `date` $PID `ls -la /proc/$PID/fd | wc -l`</span></div>
</div>
<div>
<br /></div>
<div>
Sure enough I found a leak but it was tiny and wouldn't account for the resource usage that I was seeing. I identified the source of the leak by correlating it with events on other monitoring graphs and fixed it anyway. But the graph showed something far more interesting - the file descriptors would max out consistently at ~1000 - a value far below our configured limit. I then remembered that jsvc is actually started by the root user and then drops down to the service user - perhaps it was using the open file limit for root - it looked suspiciously so:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> root@server:/ ulimit -n</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> 1024</span></div>
</div>
<div>
<br /></div>
<div>
This was exactly the same behaviour I had seen when <a href="http://swish-movement.blogspot.co.uk/2012/06/configuring-sun-jvm-to-use-huge-pages.html">configuring large pages</a> - which was unfortunately forgotten! Resource limits for jsvc based processes must be set for the user with which jsvc is started - not the user it downgrades to. This is a <a href="https://issues.apache.org/jira/browse/DAEMON-192">known issue</a> that has been raised with Apache. For a quick workaround I set the limit in my init.d script like but this feels a bit hidden and I worry that this approach would not work well when running multiple services with their own particular resource limits.</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> ulimit -n 15000</span></div>
<div>
<br /></div>
<div>
Fortunately I don't have to worry about that right now but it would be more convenient to assign a user per service and then have user based limits. Hopefully I'll now remember this jsvc behaviour so I don't get caught out by it a third time.</div>
Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com0tag:blogger.com,1999:blog-3059468813255868046.post-35415143996627829542012-10-06T13:50:00.000+01:002012-10-06T13:54:13.333+01:00Investigating unexpected GC pauses<br />
I've recently had a lot of fun learning about GC profiling. I have an service that has a fairly large heap (14G). The majority if the heap comprises an effectively static dataset that is served to clients. The application has beep performing very well using the Concurrent-Mark-Sweep collector. However, I was observing something quite strange on our dashboards - all client requests would timeout over a very short period every hour - and this occurred on every single node in the cluster. After much investigation of cron jobs, networking, etc. I decided to see what the garbage collector was doing. I had initially discounted the GC as I reasoned that CMS could not be responsible for such periodic behaviour given that pressure on memory was low and the load on the service varied throughout the day. I enabled GC logging like so:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> -verbose:gc</span><br />
<span style="font-family: Courier New, Courier, monospace;"> -Xloggc:/var/log/service-name/gc.log</span><br />
<span style="font-family: Courier New, Courier, monospace;"> -XX:+PrintGCDetails</span><br />
<span style="font-family: Courier New, Courier, monospace;"> -XX:+PrintGCDateStamps</span><br />
<br />
Unexpectedly, when the next 'timeout' period occurred I saw:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> 2012-10-05T17:20:06.498+0000: [Full GC (System) ... ]</span><br />
<br />
So, the system was doing a full 'stop the world' garbage collection cycle. Now although CMS can and will run a full GC occasionally, the periodic nature of the collections didn't quite fit - I suspected something was calling <span style="font-family: Courier New, Courier, monospace;">System.gc()</span> every hour. To investigate this I first hit <span style="font-family: Courier New, Courier, monospace;">Ctrl-Alt-H</span> on the method in Eclipse to find any invocations on the classpath - this didn't yield anything obvious. I then hooked up the service to a debugger, set a breakpoint on <span style="font-family: Courier New, Courier, monospace;">System.gc()</span> and waited. Now, I was hoping to find that some thread running in one of my 3rd party libraries was abusing the garbage collector, but when the breakpoint hit I was instead directed to:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">sun.misc.GC$Daemon</span><br />
<br />
This is the default garbage collector thread and hence looked like a bit of a dead end. I had a look at the code and found that the GC interval is controlled by a field <span style="font-family: Courier New, Courier, monospace;">private static long latencyTarget</span> - suspiciously this had a runtime value of 3600000ms (1 hour). So I had found the cause of the <span style="font-family: Courier New, Courier, monospace;">System.gc()</span> invocations but still did not know what was configuring the <span style="font-family: Courier New, Courier, monospace;">sun.misc.GC$Daemon</span> to operate every hour. As an initial workaround I used the following JVM option to override the direct GC invocations:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">-XX:-DisableExplicitGC</span><br />
<br />
However, this didn't make me that happy as part of our application benefits from an explicit <span style="font-family: Courier New, Courier, monospace;">System.gc()</span> call. Once a day clients are refused the service and a new 7GB dataset is loaded into memory. The process of loading the data can create a lot of garbage so it is nice to be able to run a full GC before it starts accepting clients again. The <span style="font-family: Courier New, Courier, monospace;">-XX:-DisableExplicitGC</span> option disables this so any garbage is going to sit around and has to be collected when we start serving clients again - not ideal.<br />
<br />
Thankfully my colleague <a href="https://github.com/Noddy76">James</a> found that some RMI components that we use in our JMX management component interact with <span style="font-family: Courier New, Courier, monospace;">sun.misc.GC</span>. The <span style="font-family: Courier New, Courier, monospace;">sun.rmi.transport.ObjectTable</span> class suspiciously requests a garbage collection latency of 1 hour by invoking the method <span style="font-family: Courier New, Courier, monospace;">sun.misc.GC.requestLatency</span>. Further <a href="http://javasourcecode.org/html/open-source/jdk/jdk-6u23/sun/rmi/transport/ObjectTable.java.html">examination of the class (lines 38, 274)</a> showed that this interval is configurable with the following system property:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">sun.rmi.dgc.server.gcInterval</span><br />
<br />
By setting these properties to <span style="font-family: Courier New, Courier, monospace;">sun.misc.GC.NO_TARGET</span> (<span style="font-family: Courier New, Courier, monospace;">== Long.MAX_VALUE == 9223372036854775807L</span>) the periodic call to <span style="font-family: Courier New, Courier, monospace;">System.gc()</span> is avoided. With this arcane property value I no longer need <span style="font-family: Courier New, Courier, monospace;">-XX:-DisableExplicitGC</span> and am free of the hourly application freeze. So the lesson here is to cautiously consider garbage collection side effects of any JDK or 3rd party libraries that your are using.<br />
<div>
<br /></div>
Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com1tag:blogger.com,1999:blog-3059468813255868046.post-44137310135806674722012-06-28T14:35:00.001+01:002012-06-28T14:36:19.249+01:00Configuring the Sun JVM to use huge pages on DebianI have a Java service that currently runs with a 20GB heap. I was keen to try out the <a href="http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html#PerformanceTuning">-<span style="font-family: 'Courier New', Courier, monospace;">XX:+UseLargePages</span></a> option to see how this might affect the performance of the system. I configured the Debian OS as <a href="http://www.oracle.com/technetwork/java/javase/tech/largememory-jsp-137182.html">described by Oracle</a> using appropriate shared memory and page values (these can also be calculated with a <a href="http://www.peuss.de/node/67">handy online tool</a>).<br />
<br />
However, I had great difficulty starting the JVM with the (sparsely) documented settings. I could see that the OS allocated the expected amount of memory as huge-pages. However, starting the VM with the <span style="font-family: 'Courier New', Courier, monospace;">-XX:+UseLargePages</span> option set always resulted in one of the following errors:<br />
<div style="text-align: left;">
<span style="font-family: Arial, 'Liberation Sans', 'DejaVu Sans', sans-serif;"><span style="font-size: 14px; line-height: 18px;"><br /></span></span></div>
When <span style="font-family: 'Courier New', Courier, monospace;">-Xmx</span> was almost equal to the huge page allocation:<br />
<div style="text-align: left;">
</div>
<ul>
<li><span style="font-size: 14px; line-height: 18px;"><span style="font-family: 'Courier New', Courier, monospace;">Failed to reserve shared memory (errno = 28). // 'No space left on device'</span></span></li>
</ul>
<br />
When <span style="font-family: 'Courier New', Courier, monospace;">-Xmx</span> is less than the huge page allocation:<br />
<div style="text-align: left;">
</div>
<ul>
<li><span style="font-size: 14px; line-height: 18px;"><span style="font-family: 'Courier New', Courier, monospace;">Failed to reserve shared memory (errno = 12). // 'Out of memory'</span></span></li>
</ul>
<br />
A helpful <a href="http://stackoverflow.com/a/11241585/74772">stackoverflow</a> answer suggested that I apply another kernel parameter that is seemingly not documented in any of the standard -<span style="font-family: 'Courier New', Courier, monospace;">XX:+UseLargePages</span> resources:<br />
<div>
<ul>
<li><span style="font-family: 'Courier New', Courier, monospace;">/proc/sys/kernel/shmall</span></li>
</ul>
<div>
This parameter must be set to an equivalent size as <span style="font-family: 'Courier New', Courier, monospace;">/proc/sys/kernel/shmmax</span> but expressed in 4KB pages units, so for a <span style="font-family: 'Courier New', Courier, monospace;">shmmax</span> size of 25769803776 <span style="font-family: 'Courier New', Courier, monospace;">shmall</span> should be 6291456. This solved some of my issues and I could test the correct instantiation of the JVM with:</div>
<ul>
<li><span style="font-family: 'Courier New', Courier, monospace;">java -XX:+UseLargePages -Xmx20g -version</span></li>
</ul>
<div>
However, my application is executed by <span style="font-family: 'Courier New', Courier, monospace;">jsvc</span> from <a href="http://commons.apache.org/daemon/">Apache commons-daemon</a>, and when starting it in this way the memory allocation problem remained. I hadn't expected this as the executing user had appropriate hugetlb group membership and adequate memlock limits in declared in <span style="font-family: 'Courier New', Courier, monospace;">/etc/security/limits.conf</span>:</div>
</div>
<div>
<ul>
<li><span style="font-family: 'Courier New', Courier, monospace;">appuser soft memlock 25165824</span></li>
<li><span style="font-family: 'Courier New', Courier, monospace;">appuser hard memlock 25165824</span></li>
</ul>
<div>
Now although <span style="font-family: 'Courier New', Courier, monospace;">jsvc</span> is normally started by the root user, it switches to a configured exec user. Some experimentation showed that appuser's memlock limits where not applied when <span style="font-family: 'Courier New', Courier, monospace;">jsvc</span> was switching from root to appuser and hence memory could not be allocated to the JVM. To get around this I also added root to the hugetlb group and provided some additional memlock limits for the root user:</div>
</div>
<div>
<ul>
<li><span style="font-family: 'Courier New', Courier, monospace;">root soft memlock 25165824</span></li>
<li><span style="font-family: 'Courier New', Courier, monospace;">root hard memlock 25165824</span></li>
</ul>
</div>
<div>
Now everything works as expected.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com0tag:blogger.com,1999:blog-3059468813255868046.post-54976251811820977532011-08-22T21:16:00.011+01:002011-08-22T21:56:21.710+01:00Update: Lion sleep woes solved<p>In my <a href="http://swish-movement.blogspot.com/2011/08/more-osx-lion-woes.html">previous post</a> I was complaining how my MacBook would no longer sleep. The sleep function worked well after installing Lion but then stopped working unexpectedly.<br />
It turns out that this is not an uncommon problem with many people complaining of the same issue. Advice on how to fix it varies wildly with solutions including (but not limited to): SMC reset, PRAM reset, turn off preference X, delete folder Y - there are even <a href="http://www.dragonone.com/products/macosx/pleasesleep/">3rd party application</a> to force your Mac to sleep.</p><p>However the <a href="https://discussions.apple.com/message/15790642#15790642">advice that worked for me</a> was actually fairly logical:</p><b>1.</b> Use <tt>pmset</tt> to view the power management settings: <br />
<pre>localhost:work teabot$ pmset -g
...
disksleep 10
<b> sleep 0 (imposed by 46)</b>
hibernatemode 3
...
</pre><p>The interesting thing in the output is the line that states '<tt>sleep 0</tt>' - presumably indicating that sleep mode is disabled. Further along this line we find '<tt>imposed by 46</tt>'. This is actually telling us the id of the process (PID) that is blocking sleep mode.</p><b>2.</b> List the process with the matching PID: <br />
<pre>localhost:work teabot$ ps -ef | grep 46
0 <b>46</b> 1 0 9:05pm ?? 0:00.16 /usr/sbin/cupsd -l
0 107 <b>46</b> 0 9:05pm ?? 0:00.59 ipp://nano.local.:631/printers/HP_Photosmart_C4100_series 7...
...
</pre><p>So we can see that a queued print job is inhibiting sleep mode and in this case I resolved the issue by clearing my print queue at which point my MacBook was happy to sleep once more.<br />
</p><p>It's pretty bizarre that a queued print job should disable sleep mode. The action of me closing the MacBook lid should make clear my intention of wanting the computer to sleep. If the print job is so important - perhaps a warning dialog should be issued so that I might have some idea why sleep mode will not be activated?</p><p>I assume that other types of process may inhibit sleep mode. Thankfully the <tt>pmset</tt> command includes some unexpectedly useful output to help us figure out which process is responsible. I can't imagine how long it would've taken me to find the errant print job without it.</p>Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com2tag:blogger.com,1999:blog-3059468813255868046.post-67545049503324878062011-08-22T08:22:00.007+01:002011-08-22T10:20:10.741+01:00More OSX Lion woesI having more small yet significant problems with OSX Lion. I'm running the latest version - 10.7.1 - on both my MacPro and MacBook.
<br />
<br /><b>My MacPro:</b>
<br />I find that I frequently lose the ability to resolve DNS names. This is only resolved by restarting the machine. Some Googling suggests that this issue may be due to a failing <tt>mDNSResponser</tt> process. I use this machine as a media server, so in practice I get up in the morning and try to tune some Internet radio only to find that I need to reboot my media server before I can do so. I'm going to try killing the process so that I don't have to restart the machine each time.
<br />
<br /><b>My MacBook:</b>
<br />Sometimes my MacBook just won't sleep. Closing the lid, Cmd-Alt-Eject, and the Apple menu option have no effect. It's really frustrating. Some people report that this can happen if Internet Sharing is enabled - it's not in my case. Apple's 'sleep' implementation used to be its crowning jewel. Coupled with OSX's stability I could: go months without rebooting my laptop, have a working system within seconds of opening the lid, happily sling my MacBook into a bag just moments after shutting the lid. I can't anymore...
<br />
<br />When I wake this machine from sleep - when I can actually get it to sleep that is, I often find that the wireless card is in an unknown state. This makes it impossible to connect to a wireless network or switch the wireless off. The machine seems to sort itself out after a few minutes but it's quite annoying - especially if I'm trying to look something up in a hurry.
<br />
<br /><b>Summary:</b> With this extremely poor OSX Lion release Apple has reminded us what it was like to run Windows XP on a cheap laptop in 2002.Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com0tag:blogger.com,1999:blog-3059468813255868046.post-73893597656931947462011-08-10T15:46:00.006+01:002011-08-22T22:00:40.945+01:00Inheriting Maven profilesI'm in the process of moving a number of projects from Ant/<a href="http://ant.apache.org/ivy/">Ivy</a> to Maven. Many of our projects use <a href="http://thrift.apache.org/">Thrift</a> and we use a <a href="http://maven.davidtrott.com/repository/org/apache/thrift/tools/maven-thrift-plugin/maven-metadata.xml">Maven Thrift plugin</a> to conveniently compile our Thrift IDL. Given that all of our Thrift projects have the same setup, the plugin is declared in a our parent POM.<br />
The plugin is declared in a profile that is only activated when a thrift source directory is present:<br />
<pre> <profile>
<id>uses-thrift</id>
<activation>
<file>
<exists>${basedir}/src/main/thrift</exists>
</file>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.thrift.tools</groupId>
<artifactId>maven-thrift-plugin</artifactId>
...
</pre><p>I actually have a number of plugins defined in such profiles so that they become active only when relevant source artefacts are present in the sub-project.</p><p>This all <span style="font-weight: bold;">looks</span> good, but in practice I found that the profiles were never activated in sub-modules that were being built from a parent project. It was as if the activation state was determined at the parent level, not on a per module basis.</p><p>It seems that profile inheritance does function as I expected in Maven 2. It does however work perfectly with Maven 3.</p>Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com0tag:blogger.com,1999:blog-3059468813255868046.post-62886182081789613262011-07-22T08:40:00.006+01:002011-07-22T09:15:59.659+01:00AFP failing between OSX Lion machinesI recently upgraded both my MacPro and MacBook to Lion. The upgrades went fine, but I now find that I cannot open a non-guest AFP connection between the two machines. On the client I see '<b>Connection Failed...</b>' and in the server console I find:<br /><pre><br />22/07/2011 08:39:20.350 AppleFileServer: _Assert: /SourceCache/afpserver/afpserver-581/afpserver/AFPRequest.cpp, 2005<br />22/07/2011 08:39:20.354 AppleFileServer: _Assert: /SourceCache/afpserver/afpserver-581/afpserver/AFPRequest.cpp, 2005<br />22/07/2011 08:39:20.400 AppleFileServer: Calling ReadLock on lock < 0x2a833128 > while holding WriteLock<br />22/07/2011 08:39:20.524 ReportCrash: DebugSymbols was unable to start a spotlight query: spotlight is not responding or disabled.<br />22/07/2011 08:39:20.965 com.apple.launchd: (com.apple.AppleFileServer[424]) Job appears to have crashed: Illegal instruction: 4<br />22/07/2011 08:39:21.062 ReportCrash: Saved crash report for AppleFileServer[424] version ??? (???) to /Library/Logs/DiagnosticReports/AppleFileServer_2011-07-22-083921_localhost.crash<br />22/07/2011 08:39:22.389 com.apple.launchd: (com.apple.AppleFileServer) Throttling respawn: Will start in 8 seconds<br /></pre><br />I also have a <a href="http://pastebin.com/Q1mv5PRV">crash report</a>.<br /><br />This is fairly annoying as my MacPro acts as the Time Machine backup for my MacBook. Has anyone experienced anything similar and are there any workarounds?<br /><br /><b>Update:</b> I found a work around: I removed all folders from 'System Preferences/Sharing/File Sharing/Shared folders' and I can now connect. The unfortunate side effect of this workaround is that now the whole file system on the server is shared - subject to the authenticated user's privileges.Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com5tag:blogger.com,1999:blog-3059468813255868046.post-62197024645036344422011-05-24T18:19:00.055+01:002011-05-25T16:04:03.975+01:00An ordered builder pattern<div>I was recently implementing the <a href="http://en.wikipedia.org/wiki/Builder_pattern">builder pattern</a> for some immutable objects and was taking the approach described by Joshua Bloch in his excellent book '<a href="http://java.sun.com/docs/books/effective/">Effective Java</a>'. I particularly like the readability that the pattern provides when constructing complex objects and I think this arises because each property value is contextualized by the preceding method name:</div><div><br /></div><div><span style="font-family:'courier new';font-size:small;"> Person.Builder builder = new Person.Builder();</span></div><div><span style="font-family:'courier new';font-size:small;"> Person person = builder.id(10).name("Jim").age(21)</span></div><div><span style="font-family:'courier new';font-size:small;"> .pets("cat", "dog").build();</span></div><div><span><br /></span></div><div><div>I also like working with builders in an IDE because the auto-complete process can nicely lead you through the object's construction with a few taps of <i>Ctrl+Space</i>. However, there are a few things that could be improved with the pattern:</div><div><br /></div><div><b>Handling of mandatory state</b></div><div>There are two obvious ways we can enforce mandatory properties with the pattern. Firstly we could pass mandatory property values in the builder constructor. This gives us a compile-time enforcement of mandatory properties. However, for many mandatory properties it can suffer from the problem that the pattern was meant to avoid - telescoping constructors. Imagine that the <span style="font-family: 'courier new'; font-size: small; ">id</span> and <span style="font-family: 'courier new'; font-size: small; ">name</span> are mandatory:</div><div><br /></div><div><div><span style="font-family:'courier new';font-size:small;"> Person.Builder builder = new Person.Builder(1, "Jim");</span></div><div><span style="font-family:'courier new';font-size:small;"> Person person = builder.age(21).pets("cat", "dog").build();</span></div></div><div><br /></div><div><span>The second way we can enforce mandatory properties is by checking the builder state in the </span><span style="font-family: 'courier new'; font-size: small; ">build()</span> method. This preserves the readablility API but relegates the mandatory property checking to run-time - quite a penalty.</div><div><br /></div><div><b>Redundant invocation paths</b></div><div>Normally a property setter method on a builder just returns the builder instance:</div><div><br /></div><div><span style="font-family:'courier new';font-size:small;"> public Builder id(int id) {</span></div><div><span style="font-family:'courier new';font-size:small;"> this.id = id;</span></div><div><span style="font-family:'courier new';font-size:small;"> return this;</span></div><div><span style="font-family:'courier new';font-size:small;"> }</span></div><div><span><br /></span></div><div>This means that on the returned value we can call any other builder method, including the one that we just invoked. We could - if we wanted to - chain many redundant calls that overwrite the builder state:</div><div><br /></div><div><span style="font-family:'courier new';font-size:small;"> Person person = builder.id(10).id(11).id(12).id(13).build();</span></div><div><span style="font-family: 'courier new'; "><br /></span></div><div>While this isn't a likely to occur, it shows that the path we take through the building of the object isn't incredibly directed - it would be nice if we couldn't set the <span style="font-family: 'courier new'; font-size: small; ">id</span> more than once. We could do this at run-time but it would be far better if we were prevented from even writing such code! This lack of order also degrades our IDE auto-complete experience a little - imagine a builder with 20 properties - every time we see the auto-complete suggestions we have to mentally parse a list of 20 property setters - including those we've already set.</div></div><div><br /></div><div><b>Adding order to the builder pattern</b></div><div>I had a go at addressing these (minor) short comings of the pattern and with some simple additions was able to create a builder that can lead you through the building process and enforce mandatory properties at compile-time in a nice contextualized way. The trick is to create a number of subordinate builder classes.</div><div><br /></div><div>Consider an immutable value class:</div><div><br /></div><div><div style="font-family:'courier new';font-size:small;"> public class Person {</div><div style="font-family:'courier new';font-size:small;"> private final int id; // MANDATORY</div><div style="font-family:'courier new';font-size:small;"> private final String name; // MANDATORY</div><div style="font-family:'courier new';font-size:small;"> private final int age; // OPTIONAL</div><div style="font-family:'courier new';font-size:small;"> private final Set<string> pets; // OPTIONAL</string></div></div><div style="font-family:'courier new';font-size:small;"> ...</div><div><br /></div><div>Now let's deal with the mandatory fields. It makes sense that you must first set the <span style="font-family: 'courier new'; font-size: small; ">id</span> property before doing anything else. Therefore the only possible method invocation from our builder class will be <span style="font-family: 'courier new'; font-size: small; ">id(int id)</span> - but what should this return? The next step in our builder sequence is to set the mandatory name, therefore we should return a builder class on which you can only invoke the <span style="font-family: 'courier new'; font-size: small; ">name(String name) </span>method, so we defined a subordinate builder class <span style="font-family: 'courier new'; font-size: small; ">NameRequiredFieldBuilder</span>, that only allows the name to be set and return this from the invocation of <span style="font-family: 'courier new'; font-size: small; ">id(int id)</span>:</div><div><br /></div><div style="font-family:'courier new';font-size:small;"><div> public static class Builder {</div><div> ...</div><div> public NameRequiredFieldBuilder id(int id) {</div><div> this.id = id;</div><div> return requiredName;</div><div> }</div></div><div><span style="font-family: 'courier new'; font-size: small; "> ...</span></div><div><br /></div><div style="font-family:'courier new';font-size:small;"><div> public static class NameRequiredFieldBuilder {</div><div> private final Builder builder;</div><div><br /></div><div> private NameRequiredFieldBuilder(Builder builder) {</div><div> this.builder = builder;</div><div> }</div><div><br /></div><div> public <b><i>?</i></b> name(String name) {</div><div> builder.name = name;</div></div><div style="font-family:'courier new';font-size:small;"> return <b><i>?</i></b></div><div><span style="font-family: 'courier new'; font-size: small; "> }</span></div><div><br /></div><div>So we can now change the builder invocations like so: <span style="font-family: 'courier new'; font-size: small; ">builder.id(10).name("Jim") </span>- in fact we no option but to call these methods, in this order. Let's deal with providing the builder with the the optional fields. At this point we'd like to be able to call <span style="font-family: 'courier new'; font-size: small; ">age(int age)</span>, <span style="font-family: 'courier new'; font-size: small; ">pets(String... pets)</span>, or <span style="font-family: 'courier new'; font-size: small; ">build()</span>. and we'd like to do this using the object returned by <span style="font-family: 'courier new'; font-size: small; ">name(String name)</span>. To achieve this we define a subordinate builder class for the optional fields and the <span style="font-family: 'courier new'; font-size: small; ">build()</span> method:</div><div><br /></div><div style="font-family: 'courier new'; font-size: small; "><div> public static class OptionalFieldsBuilder {</div><div> private final Builder builder;</div><div><br /></div><div> private OptionalFieldsBuilder(Builder builder) {</div><div> this.builder = builder;</div><div> }</div><div><br /></div><div> public OptionalFieldsBuilder age(int age) {</div><div> builder.age = age;</div><div> return builder.optionalFields;</div><div> }</div><div><br /></div><div> public OptionalFieldsBuilder pets(String... pets) {</div><div> for (String pet : pets) {</div><div> builder.pets.add(pet);</div><div> }</div><div> return builder.optionalFields;</div><div> }</div><div><br /></div><div> public Person build() {</div><div> return new Person(builder);</div><div> }</div><div><br /></div><div> }</div></div><div><br /></div><div>An instance of <span style="font-family: 'courier new'; font-size: small; ">OptionalFieldsBuilder</span> is then returned by <span style="font-family: 'courier new'; font-size: small; ">NameRequiredFieldBuilder.</span><span style="font-family: 'courier new'; font-size: small; ">name(String name)</span>:</div><div><span style="font-family: 'courier new'; font-size: small; "></span></div><div><br /></div><div><div style="font-family: 'courier new'; font-size: small; "><div> public OptionalFieldsBuilder name(String name) {</div><div> builder.name = name;</div></div><div style="font-family: 'courier new'; font-size: small; "> return optionalFields;</div><div><span style="font-family: 'courier new'; font-size: small; "> }</span></div></div><div><span style="font-family: 'courier new'; font-size: small; "><br /></span></div><div>So now we can call the builder property setters for the optional fields and then finally call the method to build an object instance. <a href="http://pastebin.com/CXyxh658">Full source code for this example</a>.</div><div><br /></div><div><b>Some observations</b></div><div>Breaking the builder into a number of subordinate builder classes allows us to introduce order into the build process. We can lead users of our API through the construction process, having them specify mandatory fields values first, only once, and potentially in order of importance. The modified pattern ensures that all mandatory fields are supplied at compile-time as it's not possible to invoke <span style="font-family: 'courier new'; font-size: small; ">build()</span> otherwise. The modified pattern does away with constructor supplied parameters and potential telescoping issues. Furthermore the subordinate builder classes limit method choices in our IDE's auto-complete mechanism to only the pertinent properties.</div><div><br /></div><div>We could introduce additional structure to the invocations of the setters of the optional properties, however this might lead to a proliferation of subordinate builder classes. I feel that there is no need to dictate the order in which optional properties are set, and indeed how many times they are set.</div><div><br /></div><div>From an implementation perspective, the overhead of creating subordinate builder classes for a few mandatory properties is small. A more scalable approach would be to use an annotation processor to generate the builder classes at compile time as had been done with Jan-Kees van Andel's <a href="http://code.google.com/p/make-builder">make-builder</a> tool.</div>Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com1tag:blogger.com,1999:blog-3059468813255868046.post-58198673981485047532011-05-24T18:08:00.004+01:002011-05-24T18:18:14.525+01:00Change of tackIt's been a while since I've developed anything for IOS and I'm not back working with Java again at <a href="http://www.last.fm/user/teabot">Last.fm</a>. I have been dabbling with C/C++ on some Arduino projects at the <a href="http://wiki.hackspace.org.uk/wiki/London_Hackspace">London Hackspace</a> - but other than that I'm pretty much a full-time Java developer. However, I really enjoyed writing about my programming experiences and have missed jotting my thoughts down on a this blog. Rather than start a new blog for general programming posts I've decided to repurpose this one given that it already contains some possibly useful programming content.<div><br /></div><div>Hopefully my (3) followers won't be too put out by this :-) I have updated the strap-line and description of this blog to reflect this new direction.</div><div><br /></div><div>For the record, the most successful Swish Movement post was that relating to <a href="http://swish-movement.blogspot.com/2009/05/private-properties-for-iphone-objective.html">private methods in Objective-C </a>- I still get comments on that post!</div>Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com0tag:blogger.com,1999:blog-3059468813255868046.post-17525477532027693062009-09-09T10:22:00.003+01:002009-09-09T11:04:35.875+01:00Accessing SSHFS within finderI usually need to access remote systems using SSHFS and SFTP from my Mac. Unfortunately OSX Finder does not support this by default. To make my life a little easier I used MacFUSE and created a shell script to mount the relevant file system:<br /><br /><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size: small;"> #!/bin/sh<br /> sshfs user@myhost.com:/aPath/ ~/Desktop/myhost.com.mount-point \</span></span><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size: small;"> -oauto_cache,reconnect,volname=myhost</span></span><div><br /></div><div>However, this can be a little frustrating so I was very happy when a friend introduced me to <a href="http://www.macfusionapp.org/">MacFusion</a> - a GUI based client for mounting SSHFS volumes. Try it!</div><div><br /><div><br /></div><div><br /></div></div></div>Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com1tag:blogger.com,1999:blog-3059468813255868046.post-81680389528744053492009-06-22T23:30:00.005+01:002009-06-23T15:28:50.514+01:00Dependency Injection with the iPhone SDK<div style="text-align: left;">Like many Enterpise Java developers - my first exposure to the <a href="http://en.wikipedia.org/wiki/Inversion_of_control">Inversion of Control</a> principle (IOC) was when applying the then radical Spring Framework some years ago. I believe that there was an IOC/DI micro container project being incubated at Apache Jakarta prior to Spring but it certainly didn't enjoy the same adoption rate.</div><div><div><br /></div><div>We now find that we rely heavily on the <a href="http://en.wikipedia.org/wiki/Dependency_injection">Dependency Injection</a> provided by our containers and frameworks. They take the responsibility for the monotinous, repetative, and tedious creational and configuration concerns of our classes leaving with much cleaner code. With some small amount of configuration they create our objects, configure them with appropriate values, and wire then up - injecting references into dependent objects. As developers this makes out lives much easier - we're no longer writing lots of boiler plate code - knowing that the frameworks will take care of it for us. Indeed we find the IOC principal in the latest releases of the EJB specification and in popular frameworks such as <a href="http://www.springsource.org/about">Spring</a>, <a href="http://seamframework.org/">Seam</a> and <a href="http://code.google.com/p/google-guice/">Guice</a>.</div><div><br /></div><div>But what does this have todo with iPhone development? Since starting work on a large iPhone project I have mostly concentrated my efforts on improving the data access and business logic layers of our application. However, over time I began to move towards the view tier - refactoring some of our <span class="Apple-style-span" style="font-family:'courier new';">UIViewContoller</span>s. More often than not the existing view controllers where monolithic classes servicing quite complex views. The views themselves could be broken down into logical units so I worked hard to move the various responsibilities of the view controllers into seperate classes that then became composite members of a parent controller - for example:</div><div><ul><li><span class="Apple-style-span" style="font-family:'courier new';">MyViewController</span></li></ul></div><div>Became:</div><div><ul><li><span class="Apple-style-span" style="font-family:'courier new';">MyParentViewController</span></li><li><span class="Apple-style-span" style="font-family:'courier new';">MyListViewController</span></li><li><span class="Apple-style-span" style="font-family:'courier new';">MyToolbarController</span></li><li><span class="Apple-style-span" style="font-family:'courier new';">MyDetailViewController</span></li></ul></div><div>While this neatly divided and encapsualted responsibility - it created many dependencies that needed to be satisfied. For example, the <span class="Apple-style-span" style="font-family:'courier new';">MyViewController</span> would have to construct it's sub-controllers and pass in references to UI components such as <span class="Apple-style-span" style="font-family:'courier new';">UIButton</span>s etc. so that they could then be managed by the sub-controller object. To do of this I would have to write a lot of boring, error-prone code - passing view components in to object via <span class="Apple-style-span" style="font-family:'courier new';">init</span> method arguments or properties. As well as being a little tedious this also prevents the code being as clean as it could be.</div><div><br /></div><div>Fortunately I was missing an iPhone SDK Inversion of Control trick. The Interface Builder application is commonly used to create views using a hierarchical view component model - a concept that I assume most are familiar with - you might draw a rectangular view, add a few buttons and arrange them etc. The magic begins when you specify which <span class="Apple-style-span" style="font-family:'courier new';">UIViewController </span>class should act as controller; Once the class has be set you can begin drawing connecting lines between the actual view components and the member variable names in that class - you describe you object dependencies using a visual metaphor - very cool! </div></div><div><br /></div><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHO_y2imVfCqgIKmuVUV9BGjQtrtQ5KlGnboU03_jg1TQf2thgM8yhhrLIV3P9V4ULKhl5-rxdpTvtVAuxxpxrDaUCWGrz9LZyI4EyvfID7kHaexecTOuCtogW9U9kMpd5Odej5NvRVjhU/s1600-h/cool-association.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 400px; height: 305px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHO_y2imVfCqgIKmuVUV9BGjQtrtQ5KlGnboU03_jg1TQf2thgM8yhhrLIV3P9V4ULKhl5-rxdpTvtVAuxxpxrDaUCWGrz9LZyI4EyvfID7kHaexecTOuCtogW9U9kMpd5Odej5NvRVjhU/s400/cool-association.png" border="0" alt="" id="BLOGGER_PHOTO_ID_5350512897730210306" /></a><div><br /></div><div>This object dependency configuration is stored in the view's XIB (nib) file and is used by the iPhone frameworks to instantiate and wire up your objects at runtime.</div><div><br /></div><div>While this is fairly intuitive for situations where we have views mapping to single monlithic controllers, it takes a little more work when we have composite controllers. In this scenario subsets of our view components might be managed by a class that is not and does not need to be a fully fledged <span class="Apple-style-span" style="font-family:'courier new';">UIViewController</span>. However, this sub-contoller class will be a member of a parent composite <span class="Apple-style-span" style="font-family:'courier new';">UIViewController</span> that pulls together multiple sub-controllers - each with their own distinct responsibilities for various view components within the larger view. Clearly, we would like to use Interface Builder to direct the instantiation of these sub-controllers and satisfy the referential dependencies between them, the parent controller, and the view components.</div><div><br /></div><div>We can do this in Interface Builder by using the 'Object' component from the component library. We create a new object for the NIB by dragging the Object component from the library. We can then specify the class of the object we wish to instantiate using the Inspector.</div><div><br /></div><div><span class="Apple-style-span" style="color: rgb(0, 0, 238); -webkit-text-decorations-in-effect: underline; "><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjUprkSHdyxKmZYFviVJwOgNbcjxl4wLEJLieuIo0_HkPgZWZuMh9BMRI_pVV6I5wN93fomYDMdeIcZd-8aprL8OtqgT1hlb6jFGNOPrgEAyFxyWGTYXDrMX-X5SNV5zFiM-u9x5xWPH7Sf/s400/create-object.png" border="0" alt="" id="BLOGGER_PHOTO_ID_5350518469470884466" style="display: block; margin-top: 0px; margin-right: auto; margin-bottom: 10px; margin-left: auto; text-align: center; cursor: pointer; width: 400px; height: 205px; " /></span></div><div><br /></div><div>However, this new object - created when the NIB is loaded - will be orphaned as it's not currently associated with any other objects. Lets assume that the parent controller may need to communicate with the sub-controller. In this event we need to create an <span class="Apple-style-span" style="font-family:'courier new';">IBOutlet</span> in the <span class="Apple-style-span" style="font-family:'courier new';">MyParentViewController</span> class:</div><div><br /></div><div><div><span class="Apple-style-span" style="font-family:'courier new';font-size:small;"> @interface MyParentViewController : UIViewController {</span></div><div><span class="Apple-style-span" style="font-family:'courier new';font-size:small;"> @private</span></div><div><span class="Apple-style-span" style="font-family:'courier new';font-size:small;"> IBOutlet MySubController* subController;</span></div><div><span class="Apple-style-span" style="font-family:'courier new';font-size:small;"> ...</span></div><div><br /></div><div>In addition to this lets assume that the responsibility of the <span class="Apple-style-span" style="font-family:'courier new';">MySubController</span> includes something button related and hence must have a reference to a <span class="Apple-style-span" style="font-family:'courier new';">UIButton</span> in the view. In the <span class="Apple-style-span" style="font-family:'courier new';">MySubController</span> class we need to create another <span class="Apple-style-span" style="font-family:'courier new';">IBOutlet</span> - this time for the button:</div><div><br /></div><div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=" ;font-family:'courier new', -webkit-fantasy;"> </span>@interface MySubController : NSObject {</span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=" ;font-family:'courier new', -webkit-fantasy;"> </span>@private</span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=" ;font-family:'courier new', -webkit-fantasy;"> <span class="Apple-style-span" style=" ;font-family:'courier new', fantasy;">IBOutlet UIButton* button;</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=" ;font-family:'courier new', -webkit-fantasy;"> </span>...</span></span></div><div><br /></div><div>Notice that <span class="Apple-style-span" style="font-family:'courier new';">MySubController</span> does not extend <span class="Apple-style-span" style="font-family:'courier new';">UIViewController</span> - it doesn't need to as it's responsibilities are small and may be focused purely on a subset of view components. We can now wire these objects up in Interface Builder:</div><div><br /></div><div><span class="Apple-style-span" style="color: rgb(0, 0, 238); -webkit-text-decorations-in-effect: underline; "><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg584Iljai9qblhcs-1RT7wa-zLybITMAcMBT8jktlPXhZtvuXKqmpyX-jv6fv6IS7Go_0ew5o2jGmWUk1x3GCLP5Z6wUU5GM4V40_ATfH6cJAm2alNdF8K-xpxst-v09Wt06lvifa7cArx/s400/wire-up.png" border="0" alt="" id="BLOGGER_PHOTO_ID_5350525087478848466" style="display: block; margin-top: 0px; margin-right: auto; margin-bottom: 10px; margin-left: auto; text-align: center; cursor: pointer; width: 400px; height: 241px; " /></span></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><span class="Apple-style-span"><span class="Apple-style-span">Now when the NIB is loaded the sub-controller object will be instantiated and a reference injected into the parent controller - all without any code or additional effort. With this approach we can begin to break-up those monolithic view controllers into classes that adhere to the <a href="http://en.wikipedia.org/wiki/Single_responsibility_principle">Single Responsibility Principle</a> so that our view controllers aren't doing 'too much'. In addition to this we can keep our classes nice and clean as we don't have to litter them with extra properties and <span class="Apple-style-span" style="font-family:'courier new';">init</span> method arguments to pass references to dependent objects. We can rely on the platform to satisfy these dependencies by injection according to the relationships we defined previously in Interface Builder.</span></span></div><div style="text-align: left;"><br /></div><div style="text-align: left;">Clearly the above example does not live in the real world. However, I have found that for complex views an controllers the approach works extremely well.</div></div></div>Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com0tag:blogger.com,1999:blog-3059468813255868046.post-23022367457866583572009-05-28T21:26:00.003+01:002009-05-28T21:48:57.909+01:00SQLite optimization on the iPhone<div>Our current iPhone application makes quite heavy use of SQLite. We persist the model in the database using a Data Access Object pattern built on top of my own <a href="http://swish-movement.blogspot.com/2009/05/revisiting-my-old-friend-jdbc-pt-1.html">JDBC like SQLite layer</a> and it is all still quite lightweight. CoreData whould have been prefereable but we have to be compatible with iPhone SDK versions prior to 3.0. We also use the database to rank trends using indexed and weighted keywords and some additional temporal factors. Thankfully the text data we search through on the device is pre-indexed so we don't have to worry too much about getting suitable data into the tables. However we do need the database to run queries against this data and we spotted early on that optimization would be beneficial here.</div><div><br /></div><div>I had some ideas of where we could improve and with a little Googling I found a sparse but useful <a href="http://web.utk.edu/~jplyon/sqlite/SQLite_optimization_FAQ.html#indexes">SQLite optimization FAQ compiled by Jim Lyon</a>. I quickly put together a hit list of optimizations that I'd try.</div><div><br /></div><div><span class="Apple-style-span" style="font-weight: bold;"><span class="Apple-style-span" style="font-size:large;">Use a between construct instead of LIKE</span></span></div><div>The FAQ explains that a LIKE cannot use an index and thus where possible it should be replaced with the > and < operators. For example:</div><div><br /></div><div><span class="Apple-style-span" style="font-family:'courier new';"> word LIKE 'dog%'</span></div><div><br /></div><div>Can be replaced with the more efficient:</div><div><br /></div><div><span class="Apple-style-span" style="font-family:'courier new';"> word > 'dog' AND word < 'doh'</span></div><div><br /></div><div>It's not a trivial replacement. Firstly the strings you are comparing must be of one case only. I also ran into trouble with international characters which of course don't fit nicely into the byte ranges. Furthermore you must write a little logic to generate the last character(s) of the 'upper limit term' (in this case 'dog'). Incrementing the last character is all well and good - but what about when you want to perform:</div><div><br /></div><div><span class="Apple-style-span" style="font-family:'courier new';"> LIKE 'oz%'</span></div><div><br /></div><div>I simply appended a low value character to the string to obtain the 'upper limit term' and ended up with something like: '<span class="Apple-style-span" style="font-family: 'courier new';">oz!</span>' Thankfully I could work within these limitations for our use-cases and performance was much improved.</div><div><br /></div><div><span class="Apple-style-span" style="font-weight: bold;"><span class="Apple-style-span" style="font-size:large;">Move members of IN clauses into temporary tables</span></span></div><div>Many of our queries used variable length IN clauses. This made it unfeasible to prepare and cache the resultant statements and they would be prepared fresh each time. This sucked up and incredible amount of time - we might spend a second just preparing the statement. A typical clause is shown here:</div><div><br /></div><div><span class="Apple-style-span" style="font-family:'courier new';"> AND id IN (32, 45, 67, 68, 80)</span></div><div><br /></div><div>I decided that if I moved these values into a temporary table I could use a sub-select within the IN clause and hence end up with a static statement that I could prepare once and cache:</div><div><br /></div><div><span class="Apple-style-span" style="font-family:'courier new';"> AND id IN (SELECT id FROM temporary_ids)</span></div><div><br /></div><div>In addition to this I hoped to use a PRAGMA directive described in the FAQ to move the temporary tables off of the flash disk and into memory:</div><div><br /></div><div><span class="Apple-style-span" style="font-family:'courier new';"> PRAGMA temp_store MEMORY</span></div><div><br /></div><div>However, this setting does not seem to take effect on the iPhone version of SQLite which was somewhat disappointing. That said, the restructuring of my IN clauses did provided yet another significant performance improvement. I wouldn't be surprised if the row inserts into the temp table actually take longer than the execution of a given IN clause. But in this instance I am avoiding the costly preparation of a statement on each call.</div><div><br /></div><div><span class="Apple-style-span" style="font-weight: bold;"><span class="Apple-style-span" style="font-size:large;">Order sub-queries so that smaller results are returned first</span></span></div><div>The FAQ suggests in section 5.3 that sub-queries or criteria should appear in an order such that the criteria that will exclude the most rows should appear first. I take this to mean that this (poor) example query:</div><div><br /></div><div><span class="Apple-style-span" style="font-family:'courier new';"> SELECT o.id</span></div><div><span class="Apple-style-span" style="font-family:'courier new';"> FROM owner o, pet p</span></div><div><span class="Apple-style-span" style="font-family:'courier new';"> WHERE <span class="Apple-style-span" style="font-weight: bold;">o.age > 12</span> AND p.name = 'nathan' AND p.id = o.pet_id</span></div><div><br /></div><div>Should be rewritten as:</div><div><br /></div><div><div><span class="Apple-style-span" style="font-family:'courier new';"> SELECT o.id</span></div><div><span class="Apple-style-span" style="font-family:'courier new';"> FROM owner o, pet p</span></div><div><span class="Apple-style-span" style="font-family:'courier new';"> WHERE <span class="Apple-style-span" style="font-weight: bold;">p.name = 'nathan'</span> AND o.age > 12 AND p.id = o.pet_id</span></div></div><div><br /></div><div>Because the <span class="Apple-style-span" style="font-family:'courier new';">p.name </span>criteria is far more selective than that using <span class="Apple-style-span" style="font-family:'courier new';">o.age</span>. Okay, so it's not a great example query. However, in our queries it was quite clear which criteria would be the most selective.</div><div><br /></div><div><span class="Apple-style-span" style="font-weight: bold;"><span class="Apple-style-span" style="font-size:large;">Other practices</span></span></div><div>Prior to implementing these optimizations we were using many of the best practices recommended in the FAQ including:</div><div><ul><li>Batching commands into transactions</li><li>Using indexes where appropriate and justifying the indexes with the EXPLAIN command</li><li>VACUUMing the database file before making release builds - this had a noticeable effect on the database file size but I couldn't say that it improved the performance.</li></ul></div><div><br /></div>Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com3tag:blogger.com,1999:blog-3059468813255868046.post-968616903424743142009-05-28T18:00:00.004+01:002009-05-28T20:25:23.228+01:00Never forget<div>Often when we are developing applications for the iPhone we want to enable extra 'tools' in the execution environment so we can get a better grasp of what our software is actually doing. Typically we'll use some kind of logging, and after reading <a href="http://www.cocoadev.com/index.pl?NSZombieEnabled"><span class="Apple-style-span">Adhamh Findlay's post</span></a> on the use of NSZombie I'll often have NSZombieEnabled turned on.</div><div><br /></div><div>Both of these tools will have an impact on the behaviour of our application if enabled. Logging will certainly slow the application down and NSZombie can make the app run out of memory. Therefore, we need to be conscious of these potential hazards so that we don't try to start fixing problems that are actually just artifacts of the tools. More importantly we must be certain that none of these useful developer settings make their way into released code. It's not hard to imagine that one could let logging slip into a build - at a minimum this will cost you another cycle of building a release and potentially much, much much more.</div><div><br /></div><div>We found ourselves making this mistake a couple of times with internal releases so thankfully it was only QA who complained and not customers. But having made this mistake we resolved to make it very difficult to repeat. So here's the first thing that our developers see when they start-up the application in a development mode:</div><div><br /></div><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8tZxoXycrHXm6Fzcw3udutt9Q8Xdjf-Lh-hdGBAy_qSVsE9G3sB_Ff3vi-MdnTXqxoJIXq9_yVgR-qtOMMEJgTm2jlu4aS_xSLKW7KkJGGJU14tkRqxb6uSV7DHr0gLG0f4DZUtJHt4lz/s1600-h/alert.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 400px; height: 180px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8tZxoXycrHXm6Fzcw3udutt9Q8Xdjf-Lh-hdGBAy_qSVsE9G3sB_Ff3vi-MdnTXqxoJIXq9_yVgR-qtOMMEJgTm2jlu4aS_xSLKW7KkJGGJU14tkRqxb6uSV7DHr0gLG0f4DZUtJHt4lz/s400/alert.png" border="0" alt="" id="BLOGGER_PHOTO_ID_5340908950212639858" /></a><div><br /></div><div>Sure, dismissing the alert is a little annoying but it reminds us of the particulars of the current execution environment and saves us from making a costly mistake. We can then release our software with greater confidence. From a code perspective it's lightweight - in <span class="Apple-style-span" style="">our </span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">viewDidLoad</span></span> method of our root <span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">UIViewController</span></span> we call:</div><div><br /></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"> <span class="Apple-style-span" style="">[ForgetfulDevelopersAlert showDeveloperWarningIfRequired];</span></span></span><br /></div><div><br /></div><div>In the <span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">ForgetfulDevelopersAlert</span></span> implementation we check for logging like so:</div><div><br /></div><div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""> BOOL logging = NO;</span></span></span></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""> #if LOG_LEVEL != NONE</span></span></span></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""> logging = YES;</span></span></span></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""> #endif</span></span></span></div><div><br /></div><div>And we detect for NSZombie usage with:</div><div><br /></div><div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> + (BOOL) __isNSZombieEnabled {</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> return getenv("NSZombieEnabled")</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> || getenv("NSAutoreleaseFreedObjectCheckEnabled");</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> }</span></span></span></div></div><div><br /></div><div>With these flags available it is straight forward to construct a message and show the <span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="">UIAlert</span></span></span>:</div><div><br /></div><div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""> if (logging || zombies) {</span></span></span></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""> UIAlertView* alert = [[ForgetfulDevelopersAlert alloc]</span></span></span></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""> __initAlertWithLogging:logging andZombies:zombies];</span></span></span></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""> [alert show];</span></span></span></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""> [alert release];</span></span></span></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""> }</span></span></span></div><div><br /></div><div>This can of course be extend to check for other development settings. For example I know of one particular case where an app was submitted to the AppStore and it was not noticed until it had finally been approved that only 'test' ad banners had been enabled. Potentially this lost some revenue and required another approval cycle with Apple - but it is a very easy mistake to make that perhaps could have been avoided with something like:</div><div><br /></div><div><span class="Apple-style-span" style=" ;font-family:'courier new';font-size:13px;"> <span class="Apple-style-span" style="">[ForgetfulDevelopersAlert showWarningIfTestAdsEnabled];</span></span><br /></div><div><span class="Apple-style-span" style=" ;font-family:'courier new';font-size:13px;"><br /></span></div></div></div><div>:-)</div><div><br /></div>Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com1tag:blogger.com,1999:blog-3059468813255868046.post-89759594915675236832009-05-27T13:05:00.007+01:002009-05-28T20:20:02.170+01:00Private properties for iPhone Objective-C<div>Keeping members private is extremely important in Object Oriented design as it allows us to adhere to the principles of <a href="http://en.wikipedia.org/wiki/Encapsulation_(computer_science)"><span class="Apple-style-span">encapsulation</span></a> and <a href="http://en.wikipedia.org/wiki/Information_hiding"><span class="Apple-style-span">information hiding</span></a>, and makes it easier to keep our objects thread safe. We often want to use @properties in Objective-C to take care of retain counts in setters for us. However, unless you are careful these properties will always be public. This 'kitchen sink' approach results in all of your internal object state being exposed on your public API. This is dangerous because:</div><div><ul><li>Developers can modify the internal state of your objects when they shouldn't<br /></li><li>Developers are 'tempted' to access things that they shouldn't as they can see 'apparently useful' things on the interface.<br /></li><li>Developers are confused when they read the interface - they need to somehow work out which public properties they can use and which they can't.</li></ul></div><div>In short: <span class="Apple-style-span" style="font-weight: bold;">NEVER</span> expose class internals publicly - it is very bad OO practice.</div><div><br /></div><div>Private setters are discussed in <a href="http://theocacao.com/document.page/516"><span class="Apple-style-span">Scott Stevenson's Objective-C tutorial</span></a>, but the post does not explain how to make both the setter and getter private - furthermore I was tripped up by a coupled of points that are not illustrated in Scott's examples. So how do we create completely private properties in Objective-C? The steps are as follows:</div><div><ol><li>Make the member variable @private. This is not essential to create the property but if we're making the property private then it'd be wise to make the member variable accessible only from within our class.<br /></li><li><span class="Apple-style-span" style="font-weight: bold;">DO NOT</span> declare the property in the main @interface in the header (.h) file<br /></li><li>Create an @interface for an extension (unnamed category) in the .m file <span class="Apple-style-span" style="font-weight: bold;">ABOVE</span> the @implementation<br /></li><li>Declare your private @property in the extension interface<br /></li><li>@synthesize the private property in your @implementation as normal.</li></ol></div><div>See code below for an example:</div><div><br /></div><div>ClassWithPrivateProperty.h</div><div><br /></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> @interface ClassWithPrivateProperty : NSObject {</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> </span><span class="Apple-style-span" style="font-weight: bold;"><span class="Apple-style-span" style="">@private</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> NSString* member;</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> }</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> - (void) trySettingPrivateProperty;</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> @end</span></span></span></div><div><br /></div><div>ClassWithPrivateProperty.m</div><div><br /></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> #import "ClassWithPrivateProperty.h"</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> </span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-weight: bold;"><span class="Apple-style-span" style=""> @interface ClassWithPrivateProperty ()</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-weight: bold;"><span class="Apple-style-span" style=""> @property (nonatomic,retain) NSString* member; </span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-weight: bold;"><span class="Apple-style-span" style=""> @end</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> </span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> @implementation ClassWithPrivateProperty</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> </span><span class="Apple-style-span" style="font-weight: bold;"><span class="Apple-style-span" style="">@synthesize member;</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> - (void) trySettingPrivateProperty {</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> self.member = @"A Value";</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> NSLog(@"myClass.member = %@", self.member);</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> }</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> @end</span></span></span></div><div><br /></div>Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com17tag:blogger.com,1999:blog-3059468813255868046.post-13680938998847950532009-05-25T10:36:00.023+01:002009-05-28T21:31:49.533+01:00A simple concurrency framework for the iPhone<div>The iPhone app we're working on has become quite large and complex - necessarily so. However, from the start of development many months ago the product has grown organically. Although almost feature complete - the app was quite unstable and failed to deliver in terms of performance so some refactoring was needed.</div><div><br /></div><div>As I worked my way through spaghetti code, I came across many threading objects and asynchronous calls that were initiated and 'managed' all over the application, sometimes in the view, sometimes the model, and sometimes on the controllers. This effectively made it impossible to determine what the application was doing at any given time. To make matters worse it it was not clear that we were being thread-safe as access to some shared data was unsynchronized. These findings were the likely root of our stability issues. In addition to this we found some tight thread-sleep loops. These were a possible candidate for some performance issues but also a total abomination.</div><div><br /></div><div>To tackle these issues I had a plan. I would create a framework that provided the threading functionality required by our application use-cases in a safe and centralized manner. I'll now describe the framework.</div><div><br /></div><div>The foundation of the framework is an old favourite - the <a href="http://en.wikipedia.org/wiki/Command_pattern"><span class="Apple-style-span">command pattern</span></a>. Any piece of work we want to do on a thread - be it make an HTTP call, parse a document, or asynchronously write to the database - is expressed as a command. In practice we must implement a protocol:</div><div><br /></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"> @protocol Command</span></span></span></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"> - (void) execute;</span></span></span></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"> @end</span></span></span></div><div><br /></div><div>Concrete command classes usually pass some input parameters in the constructor and provide an accessor so that the caller can extract any result of the processing that takes place in the execute method. The execute method implementation is the 'stuff' you actually want to DO be it synchronously or asyncrhonously.</div><div><br /></div><div>Fundamentally the framework executes commands. However, regardless of your concrete command implementation - the framework can execute your command in a synchronous (blocking) or asynchronous manner. What a relief for the developers - forget about how we safely do something asynchronously and concentrate on the thing you actually want to DO. The framework provides the following methods on the core class - <span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="">CommandSchedulerQueue</span></span></span> - for command execution:</div><div><br /></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> - (void) <span class="Apple-style-span" style="font-weight: bold;">addCommandToQueueAndReturn</span>:(id<Command></span><command><span class="Apple-style-span" style="">)command;</span></command></span></span></span></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> - (void) <span class="Apple-style-span" style="font-weight: bold;">addCommandToQueueAndReturn</span>:(id</span><command><span class="Apple-style-span" style=""><Command>)command <span class="Apple-style-span" style="font-weight: bold;"></span></span></command></span></span></span></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""><command><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-weight: bold;"><span class="Apple-style-span" style="font-weight: normal; "> <span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-weight: bold;">withCallback</span>:(id</span><mutablecallback><span class="Apple-style-span" style=""><MutableCallback>)callback;</span></mutablecallback></span></span></span></command></span></span></span></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> - (void) <span class="Apple-style-span" style="font-weight: bold;">addCommandToQueueAndWait</span>:(id</span><command><span class="Apple-style-span" style=""><Command>)command <span class="Apple-style-span" style="font-weight: bold;"></span></span></command></span></span></span></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""><command><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-weight: bold;"><span class="Apple-style-span" style="font-weight: normal; "> <span class="Apple-style-span" style="font-weight: bold;">withTimeoutInSeconds</span>:(NSInteger)timeout;</span></span></span></command></span></span></span></div><div><br /></div><div>As the method names suggest - the framework provides a command queue internally which mandates that the commands be executed in the sequence that they were added to the queue. This may sound a little restrictive but it is perfectly possible to create multiple command queues or even allow more than one command to be executed concurrently on a given queue by allocating the queue more threads.</div><div><br /></div><div>So before I describe the functionality given to commands and callbacks by the framework, let me mention what's going on inside the framework - and specifically the thing we call the <span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';">CommandSchedulerQueue</span></span></span> class. From the outset I wanted to stick to using well understood Foundation components and classes. Indeed the Foundation framework provides a <a href="http://developer.apple.com/documentation/Cocoa/Conceptual/Multithreading/index.html"><span class="Apple-style-span">rich set of classes to deal with concurrency</span></a> - so use them! At the core of the <span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="">CommandSchedulerQueue</span></span></span> class is an <span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';">NSOperationQueue</span></span></span>, we wrap our command execution in an <span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';">NSOperation</span></span></span> and schedule it on the operation queue. This is nothing new - our application used <span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';">NSOpeationQueue</span></span></span>s in the past but often created them all over the place and applied them in an inconsistent manner. By wrapping an <span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">NSOperationQueue</span></span></span> in our <span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">CommandSchedulerQueue</span></span></span> we can add useful behavior and hide <span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">NSOperationQueue</span></span></span></span> specifics that previously tempted developers to do undesirable things.</div><div><br /></div><div>So what kind of behavior do we add to the command execution?</div><div><ul><li>We can block the caller until the operation - or rather the command - is complete. We do this in a safe and performant manner using the <span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';">NSCondition</span></span></span> class.<br /></li><li>We can invoke a callback to the caller if they have scheduled a command to be executed asyncronously.<br /></li><li>We can capture an exception raised during the command's execution and either throw it back to the caller immediately if the command was executed synchronously, or we can deliver the exception in the callback in the case of asynchronous execution.</li></ul></div><div>Let's take a look at the callback:</div><div><br /></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> @protocol Callback</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> - (void) </span></span><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-weight: bold;">callbackInvokedF</span></span></span><callback><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-weight: bold;">orCommand</span>:(id</span></span><command><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><Command></span></span><span class="Apple-style-span" style=""><callback><command><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="">)command</span></span></command></callback></span><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="">;</span></span></command></callback></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> - (void) </span></span><span class="Apple-style-span" style="font-weight: bold;"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="">throwExecutionException</span></span></span><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="">;</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> @end</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><br /></span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> @protocol MutableCallback <Callback></span></span><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><callback></callback></span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> - (void) setExecutionException:(NSException*)exception;</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> @end</span></span></span></span></div><div><br /></div><div>When implementing the callback for a command we need to provide the behavior of the <span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="">callbackInvoked</span></span></span> method - i.e what we want to happen when the callback is finally fired. Often we'll want to send a message to the original caller to tell them that their command is no longer executing. The receiver of the callback can then invoke <span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="">throwExecutionException</span></span></span> to handle any exception that occurred during the execution of the command. An implementation of <span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">throwExecutionException</span></span></span> is actually provided in an abstract <span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';"><MutableCallback></span></span></span> implementation and the <span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';">CommandSchedulerQueue</span></span></span> knows how to set the exception within the callback object.</div><div><br /></div><div>So that's the basic framework. In our iPhone application we now put all concurrent functionality through this framework which allows us to keep the threading complexity in one place and provides us with excellent visibility on what the app is doing at any given time. We can schedule commands knowing that they'll be executed in sequence and with the confidence that we won't miss any exceptions thrown - even if they're thrown on another thread. Finally we can schedule operations and block until completion in a clean manner (no tight thread-sleep loops!) while hiding the specifics of <span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'courier new';">NSCondition</span></span></span>.</div><div><br /></div>Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com1tag:blogger.com,1999:blog-3059468813255868046.post-5553629146975774672009-05-03T16:32:00.000+01:002009-05-03T16:12:43.884+01:00Code as prose<div>I recently started reading <a href="http://www.amazon.co.uk/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882">Clean Code</a> - and immediately felt guilty for not reading it earlier. In the first few pages Grady Booch suggests that clean code should read like prose. If we take this a little further use an analogy where the code of an application could be thought of as a book we could then ask the question: '<span class="Apple-style-span" style="font-style: italic;">How would the the shelves of our local bookshop appear?</span>' Their contents would surely reflect the current state and quality of IT projects at large.</div><div><br /></div><div>Personally I think there would be a fair few fine works of prose and many of those you could take home for free. However, the majority of those shelves would be chock full of vaguely similar <a href="http://www.flickr.com/photos/timboduke/2657395815/">children's colouring-in books that have been carelessly scribbled in</a>. Markings from the 'crayon of implementation' would wildly stray over the clear and purposeful black outlines of original intent.</div><div><br /></div><div>Flickr is surprisingly devoid of badly executed 'colouring-ins' but has no shortage of perfectly filled pages from such books - perhaps this is yet another example of <a href="http://www.nysscpa.org/cpajournal/2003/1203/nv/nv2.htm">sunlight being the best disinfectant</a>. </div>Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com1tag:blogger.com,1999:blog-3059468813255868046.post-33380898766351517372009-05-03T08:58:00.022+01:002009-05-28T20:24:36.565+01:00Revisiting JDBC (pt. 1)About 6 or 7 years ago I was writing a lot of SQL and <a href="http://java.sun.com/javase/6/docs/technotes/guides/jdbc/">JDBC</a> - I remember being particularly pleased when I developed an efficient implementation of <a href="http://www.intelligententerprise.com/001020/celko.jhtml">Celko's Nested sets</a> to represent hierarchical data in a Content Management project. At the time JDBC was a pretty neat way of interfacing with a relational database. However, we Enterprise Java developers have for the most part left JDBC behind in favor of excellent ORM frameworks such as <a href="http://www.hibernate.org/">Hibernate</a> and although we are using JDBC more that ever, we do so with it operating under the covers - tucked away within our ORM framework. Sure there may be times when we have to step back to JDBC - how well would Hibernate handle hierarchical data? - but they are infrequent.<div><br /></div><div>Now as I mentioned in previous posts, the main reason I found myself developing - well, debugging at first - on the iPhone platform was to investigate why a SQLite based feature of our application wasn't functioning as well as the original Java proof-of-concept. Fairly soon I was immersed in the world of <a href="http://www.sqlite.org/c3ref/intro.html">SQLite's C/C++ interface</a> - JDBC this was not. From a Java point of view it's low-level: Error conditions signaled by return value on almost every function and pointers aplenty. What I wanted was JDBC - or rather <span class="Apple-style-span" style="font-style: italic;">OBJC</span>DBC - but in fact I wanted much more because in the JDBC world I had also become used to:</div><div><ul><li>Excellent connection and prepared statement pooling with the likes of <a href="http://commons.apache.org/dbcp/">DBCP</a>.</li><li>Concise utility methods that allowed me to avoid JDBC boiler-plate in the form of <a href="http://commons.apache.org/dbutils/">DbUtil</a>.</li></ul><div>I Googled a while to see if such a thing existed in the iPhone domain and thankfully found many likely candidates on the <a href="http://www.sqlite.org/cvstrac/wiki?p=SqliteWrappers">SQL Wiki</a> (see: Objective-C section) ranging from simple wrappers to ORM frameworks. However, the simple wrappers were not as clean and object orientated as I was used to with JDBC and the ORM frameworks would not give me enough control to write some of the highly optimized SQL queries that our application demanded. I set about writing my own wrapper. One could argue that I was <a href="http://www.codinghorror.com/blog/archives/001145.html">reinventing the wheel</a> - but I'm always happy to learn more about wheels. My requirements were as follows:</div><div><ul><li>Option to pool connections and statements</li><li>Check for <span class="Apple-style-span" style="font-weight: bold;">every</span> error condition that SQLite could possibly set and convert these into exceptions so we can adopt a <span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">try</span></span>/<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">catch</span></span>/<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">finally</span></span> approach when accessing the database.</li><li>Provide a clean and simple programmatic interface.</li><li>Move all direct interactions with SQLite into a few sensible classes - C based SQLite code had previously been spread liberally throughout the application.</li></ul><div>My core classes were to be as follows:</div><div><ul><li><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">PooledDataSource</span></span> - A connection data source that also pools connections. Calling close on a connection actually returns it to the pool.<br /></li><li><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">Connection</span></span> - Encapsulates a SQLite database handle - can also pool statements that have been prepared from this connection. Calling prepare on a connection might actually fetch and reuse a pooled <span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style="">PreparedStatement</span></span> rather than creating a new instance. rovides methods to manage transactions.</li><li><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">PreparedStatement</span></span> - Prepares transient and non-transient statements, binds parameter values to statements, and executes statements. Returns <span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">ResultSet</span></span>s for <span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">SELECT</span></span> queries. Calling close on a prepared statement returns it to the statement pool if it is not transient.</li><li><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';">ResultSet</span></span> - An interface for stepping through a cursor and retrieving values from the row.</li></ul><div>With these classes in place I could write some typical Data Access Object code:</div><div><br /></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> PreparedStatement statement;</span></span></span></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> ResultSet results;</span></span></span></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> NSNumber maxId = nil;</span></span></span></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> @try {</span></span></span></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> stmt = [con prepare:@"SELECT MAX(id) FROM foo WHERE bar = ?"];</span></span></span></div><div><span class="Apple-style-span" style=";font-family:'courier new';"> [con begin]; // OK - so we <span class="Apple-style-span" style="font-weight: bold;">don't need</span> this for a SELECT</span></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> results = [stmt executeWithValues:@"dog", nil];</span></span></span></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> if ([results next]) {</span></span></span></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> maxId = [results getInt:1];</span></span></span></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> }</span></span></span></div><div><span class="Apple-style-span" style=";font-family:'courier new';"> [con commit]; // for illustration <span class="Apple-style-span" style="font-weight: bold;">only</span></span></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> @catch (NSException* sqlX) {</span></span></span></div><div><span class="Apple-style-span" style=";font-family:'courier new';"> [con rollback]; // for illustration <span class="Apple-style-span" style="font-weight: bold;">only</span></span></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> // handle error</span></span></span></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> } @finally {</span></span></span></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> <span class="Apple-style-span" style=""> </span>[DatabaseCommons closeQuietly:results, statement, con];</span></span></span></div><div><span class="Apple-style-span" style=""><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> }</span></span></span></div><div><br /></div></div></div><div>I'll get more into the actual class implementation details in part 2.</div><div><br /></div><div><br /></div></div>Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com0tag:blogger.com,1999:blog-3059468813255868046.post-59687829369453304642009-05-02T17:10:00.002+01:002009-05-28T20:23:05.355+01:00Dynamic Proxies<div>Part of our iPhone application uses proxy objects lazily load the full-fat objects behind the scenes. The initial implementation simply used an object that had the same core interface of the proxied object and used a factory to load the full-fat object in the event that a property was called on the proxy that it couldn’t service with it’s own internal state.<br /></div><div><span class="Apple-style-span" style="font-size:x-small;"><br /></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> @implementation MyProxyClass // : NSObject ... </span></span></span><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""><myprotocol></myprotocol></span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> -</span></span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="">(NSString*)doSomethingIntensive {</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="white-space: pre;"><span class="Apple-style-span" style="white-space: normal; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span></span><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span></span><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="">if ([self fullFatObjectNotLoaded]) {</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> [self loadFullFatObject];</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> }</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> return [fullFat doSomethingIntensive];</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> }</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> ...</span></span></span></span></div><div><br /></div><div>This was all well and good - but eventually the model sitting behind the proxy grew and grew until we had a rather large hierarchy of classes that had to be managed by the proxy. To do this the proxy would have to present all of the varied properties on all of the classes in the model hierarchy. In development terms alone this wasn’t scalable - we’d keep having to add new properties to the proxy as they were added to the model; instead a shortcut around this issue had been implemented. The proxy exposed the full-fat object with it’s own getter:</div><div><br /></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> @interface MyProxyClass : NSObject ...</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> -</span></span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="">(MyClass*)getFullFat;</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> ...</span></span></span></span></div><div><br /></div><div>The problem with this is of course that now the caller to the proxy must actually know about the proxy - it can no longer just use the MyProtocol protocol (<span class="Apple-style-span" style="">Java: ‘Interface’</span>) and must now be aware of the type ‘<span class="Apple-style-span" style="font-family:'courier new';">MyProxyClass</span>’. This makes the calling code tightly coupled to the lazy loading process - exactly what we don’t want.</div><div><br /></div><div>As enterprise Java developers we see dynamic proxies intercepting calls to our classes everywhere in our EJB containers, Dependency injection frameworks (e.g: Spring, Seam), and ORM frameworks (Hibernate for example). In truth we rarely see them - perhaps in a call stack in the debugger - but we know they are there wrapping rich services around our objects without our intervention.</div><div><br /></div><div>What I wanted was something similar for our iPhone application - some kind of Objective-C dynamic proxy that could seamlessly proxy our (perhaps overly) complex model and remain hidden from the classes who would actually be be calling the proxy.</div><div><br /></div><div>Now writing a dynamic proxy service in Java is probably difficult. I haven’t written one but I imagine it entails reflection and byte code generation - not necessarily the stuff that the average Java developer gets stuck into. Besides - there are good libraries that do this already and do it well in the Java world. However - in the world of Objective-C it is surprisingly easy to do out of the box using a Foundation class named ‘<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">NSProxy</span></span>’.</div><div><br /></div><div>Any selector messages (<span class="Apple-style-span" style="">Java: ‘method calls’</span>) invoked on the proxy invoke a default method on the proxy called ‘<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">forwardInvocation</span></span>’ including a description of the invocation (selector, arguments, etc.) By overriding the ‘<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">forwardInvocation</span></span>’ method we can route the method invocation wherever we like. In the case of our lazy loading proxy we’d like to check to see if our proxy can service the invocation (we can use '<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">respondsToSelector</span></span>' for this) and if not - load up the object we are the proxy for and then forward the invocation to that full fat object like so:</div><div><br /></div><div><span class="Apple-style-span" style="white-space: pre; font-family:'courier new';"><span class="Apple-style-span" style="white-space: normal; "> @implementation MyProxyClass // : NSProxy ...</span><br /></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="white-space: pre; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span></span><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="">-(void)forwardInvocation:(NSInvocation*)invocation {</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span><span class="Apple-style-span" style="white-space: pre; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span><span class="Apple-style-span" style="white-space: normal; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="">SEL selector = [invocation selector];</span></span></span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="white-space: pre;"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span><span class="Apple-style-span" style="white-space: normal; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="">if ([self fullFatNotLoadedAndCanBeHandledByProxy:selector]) {</span></span></span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="white-space: pre;"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span><span class="Apple-style-span" style="white-space: normal; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="">[invocation setTarget:lowFat];</span></span></span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="white-space: pre; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span><span class="Apple-style-span" style="white-space: normal; "><span class="Apple-style-span" style="white-space: pre; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span></span><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="">} else if (fullFat != nil) {</span></span></span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="white-space: pre; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span><span class="Apple-style-span" style="white-space: normal; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="">[invocation setTarget: fullFat];</span></span></span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="white-space: pre; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span><span class="Apple-style-span" style="white-space: normal; "><span class="Apple-style-span" style="white-space: pre; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span></span><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="">} else {</span></span></span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="white-space: pre; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span><span class="Apple-style-span" style="white-space: normal; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="">NSAssert(FALSE, @"Hope this never happens");</span></span></span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="white-space: pre; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span><span class="Apple-style-span" style="white-space: normal; "><span class="Apple-style-span" style="white-space: pre; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span></span><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="">}</span></span></span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="white-space: pre; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span><span class="Apple-style-span" style="white-space: normal; "><span class="Apple-style-span" style="white-space: pre; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span></span><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="">[invocation invoke];</span></span></span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="white-space: pre; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span><span class="Apple-style-span" style="white-space: normal; "><span class="Apple-style-span" style="white-space: pre; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span></span><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="">return;</span></span></span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="white-space: pre; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> </span></span></span></span><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style="">}</span></span></span></span></div><div><br /></div><div>Of course - it’s a bit more complicated than that. <span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">NSProxy</span></span> is not an actual object as such - it’s not a subclass of <span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">NSObject</span></span>. So if you send an ‘init’ invocation to a <span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">NSProxy</span></span> instance it doesn’t actually call the <span class="Apple-style-span" style="font-family:'courier new';">init</span> (<span class="Apple-style-span" style="">Java: ‘default constructor’</span>) on the proxy instance - it doesn’t have one. Instead this is treated like an other method and we find that we have an invocation of ‘<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">forwardInvocation</span></span>’ with selector ‘<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">init</span></span>’. And in this event what object reference should we return to the caller? It gets even more confusing when we find that calls to extremely type specific methods such as: ‘<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">isKindOfClass</span></span>’, ‘<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">conformsToProtocol</span></span>’, and ‘<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">respondsToSelector</span></span>’ also get directed straight to ‘<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">forwardInvocation</span></span>’ and that we have to handle those too! How confusing - this would never happen in Java!</div><div><br /></div><div>So this is the way that I chose to implement the lazy loading proxy:</div><div><ul><li>The proxy would have an internal ‘low-fat’ skeleton object that acts as a placeholder for the proxied object - in my case it had a member that held the full fat objects database ID. The low fat object is alloced when the proxy gets a request for the <span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">NSMethodSignature</span></span> for the <span class="Apple-style-span" style="font-family:'courier new';">initWithId</span> method - the proxy passes a reference to it<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">self</span></span> - not the databaseId - we have to wait for the '<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">forwardInvocation</span></span>' call to get at the method call argument.<br /></li></ul><div><div><span class="Apple-style-span" style=";font-family:'courier new';"> @implementation MyProxyClass // : NSProxy ...</span></div><div><span class="Apple-style-span" style=";font-family:'courier new';"> ...</span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> </span></span><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> -(NSMethodSignature*)methodSignatureForSelector:(SEL)selector {</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> if (selector == @selector(initWithId:)) {</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> if (lowFat == nil) {</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> lowFat = [[LowFatObject alloc] initWithProxy:self];</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> } else {</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> NSAssert(FALSE, @"Hope this never happens");</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> }</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> }</span></span></span></span></div><div><span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> ...</span></span></span></span></div><div><span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> }</span></span></span></span></div><div><span class="Apple-style-span" style=";font-family:'courier new';"> ...</span></div><div><span class="Apple-style-span" style=";font-family:'courier new';"> @end</span></div><div><span class="Apple-style-span" style=";font-family:'courier new';"><br /></span></div></div><div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> @implementation LowFatObject</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> @synthesize databaseId;</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> -(id)initWithProxy:(MyProxyClass*)aProxy {</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> if (self = [super init]) {</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> proxy = aProxy;</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> // NOTE: 'proxy' member of type MyProxyClass!</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> }</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> return self;</span></span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><span class="Apple-style-span" style=""> }</span></span></span></span></div></div><ul><li>The ‘<span class="Apple-style-span" style="font-family:'courier new';">initWithId</span>’ call to the proxy would actually be routed to the low-fat object to but note that the low fat object has already been alloced and init'ed by the proxy. So instead it adopts the databaseId value and (importantly) returns a reference to the <span class="Apple-style-span" style="font-style: italic;">proxy</span> - <span class="Apple-style-span" style="font-weight: bold;">not</span> it<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">self</span></span>.<br /></li></ul><div><span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> @implementation LowFatObject</span></span></span><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""><br /></span></span></div><div><span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> -(id)initWithProxy:(MyProxyClass*)aProxy...</span></span></span></div><div><span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> ...</span></span></span></div><div><span class="Apple-style-span" style=" ;font-family:'courier new';font-size:13px;"><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> -(id)initWithId:(int)aDatabaseId {</span></span></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> databaseId = aDatabaseId;</span></span></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> return <span class="Apple-style-span" style="font-weight: bold;">proxy</span>;</span></span></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> // We want the caller to have a ref to the <span class="Apple-style-span" style="font-weight: bold;">proxy</span>!</span></span></div><div><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> }</span></span></div></span></div><ul><li>We're still left with the issue of deciding when to load the full-fat item. To do this we need to check if the low-fat skeleton object is able to service the invocation and if not we need to load the full-fat object and set that as the invocation target. We do this when we're asked to supply the '<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">NSMethodSignature</span></span>' for the pending invocation. Before we dive into this recall that the invocation could be for something where the return type is extremely type specific - such as '<span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="">respondsToSelector</span></span>' - in those eventualities we MUST also load the full-fat object because only the can we really know what the object we are a proxy for is actually capable of.</li></ul></div><div><div><span class="Apple-style-span" style=";font-family:'courier new';"><span class="Apple-style-span" style=";font-family:Georgia;"><div><span class="Apple-style-span" style=";font-family:'courier new';"> @implementation MyProxyClass // : NSProxy ...</span></div><div><span class="Apple-style-span" style=";font-family:'courier new';"> ...</span></div></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> -(NSMethodSignature*)methodSignatureForSelector:(SEL)selector {</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> ... init method handler ...</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> NSObject* target;</span></span></span></div><div><span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> // Is the selector extremely type specific?</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> if (selector == @selector(isKindOfClass:)</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> || selector == @selector(conformsToProtocol:)</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> || selector == @selector(respondsToSelector:))</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> {</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> [self loadFullFatTarget];</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> target = fullFat;</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> } else if ([self fullFatNotLoadedAndCanBeHandledByProxy:selector]) {</span></span></span></div><div><span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> // The low-fat item responds to the selector</span></span></span></div><div><span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> // and the full-fat hasn't been loaded</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> target = lowFat;</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> } else {</span></span></span></div><div><span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> // We need to load the full fat because the low-fat</span></span></span></div><div><span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> // skeleton can't handle it</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> [self loadFullFatTarget];</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> NSAssert1(</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> [fullFat respondsToSelector:selector],</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> @"Neither full/low fat object can handle selector %@",</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> selector</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> );</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> target = fullFat;</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> }</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> return [target methodSignatureForSelector:selector];</span></span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=""> }</span></span></span></div><div><br /></div><div>Despite being a little fiddly this approach has worked quite well in practice. As a Java developer it's quite refreshing to be able to construct something as powerful as a Dynamic Proxy using a core class supplied in the SDK. Although the model we are proxying probably has much room for refactoring we can now at least add and remove properties from the model classes and have them implicitly supported by the proxy without any further modification to the proxy itself.</div><div><br /></div></div>Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com0tag:blogger.com,1999:blog-3059468813255868046.post-90388356684293303572009-05-02T16:59:00.001+01:002009-05-03T10:44:57.307+01:00From (My)Eclipse to XCodeAs any Java developer worth his salt knows - if you’re going to be writing code in a productive way - you should at least know a handful of keyboard shortcuts in Eclipse. Yes I’m sure others use Netbeans and IntelliJ IDEA - but I have met them rarely and they often seem to have selected their IDE just to be different, awkward, or both (flame war starts here).<br /><br />So important is the knowledge of shortcuts that I’ve often seen them on cheat sheets, stuck on the side of mugs, and described incredulously in code reviews when one developer who knows a shortcut, watches the other developer painfully scroll through a menu or type out a class name. Some developers - so happy that they’ve discovered something powerful - email all of their developer friends with the newly learnt shortcut only to be ridiculed in a barrage of replies with sentiments to the effect of: ‘You’ve been earning £X00 a day developing with Eclipse for 5 years and you didn’t know about shortcut X?’<br /><br />So anyway, my first big hurdle with XCode was shortcuts. Without those I could not be as productive as I’d like. In fact - that wasn’t my first hurdle - first I needed to stop using Eclipse shortcuts in XCode.<br /><br />So I’ll list my favorite XCode shortcuts. They are there in the preference panels, in books and on the side of mugs. But if like me you can’t be bothered to look and would rather have someone else tell you here they are:<br /><ul><li><span class="Apple-style-span" style="font-weight: bold;">alt-cmd-up</span>: Switch between header and implementation file<br /></li><li><span class="Apple-style-span" style="font-weight: bold;">alt-cmd-left</span>: go to previous file<br /></li><li><span class="Apple-style-span" style="font-weight: bold;">alt-cmd-right</span>: go to next file<br /></li><li><span class="Apple-style-span" style="font-weight: bold;">cmd-b</span>: start/stop build<br /></li><li><span class="Apple-style-span" style="font-weight: bold;">shift-cmd-k</span>: clean all targets<br /></li><li><span class="Apple-style-span" style="font-weight: bold;">cmd-return</span>: build and go (launches the iPhone simulator/device with your app)<br /></li><li><span class="Apple-style-span" style="font-weight: bold;">shift-cmd-d</span>: find a source file<br /></li><li><span class="Apple-style-span" style="font-weight: bold;">shift-cmd-f</span>: find text in the project sources<br /></li></ul>Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com0tag:blogger.com,1999:blog-3059468813255868046.post-24593211939182233392009-05-02T16:52:00.002+01:002009-05-03T11:08:28.562+01:00BackgroundSo a little bit of background. I have been happily developing with Java for over 9 years and am very appreciative of the wealth of frameworks available that make my job easier and more productive. In the company I work for currently I have been developing an enterprise server side Java application whose job is to aggregate and feed data to an iPhone based client. It got to the point where the server was virtually complete, scaling well and had few additional features remaining in the product backlog. We ‘server guys’ had applied best practices, patterns, and frameworks with a very satisfactory result.<br /><br />I then began architecting features of the iPhone client application - with no intention of actually developing on the platform but merely providing the iPhone developers with a proof of concept that they could effectively ‘transcribe‘ into the iPhone domain. The feature in question required an SQLite database on the iPhone and as a server guy I could throw SQL together with ease - so it made sense that I should do the design. However, as the iPhone implementation of my designs progressed, many issues presented themselves on the client that just didn’t exist in my Java-based proof of concept. It was at this point I realized that I’d have to delve into the world of Objective-C and XCode. Having done a bit of C programming at university and written one MFC based OpenGL app for my coursework 12 years previous - I was struck with fear at the learning curve I probably faced.<br /><br />I was very pleasantly surprised. In terms of language structure and syntax - the core things that make Java development easier (or rather: less difficult) than C were all in place in Objective-C. Exceptions, Thread classes, synchronization, interfaces, etc. were there - although often they are often called something totally different and aren’t quite like the Java equivalent. I was given Dylan McNamee’s ‘<a href="http://www.mecodegoodsomeday.com/ObjectiveCRef.pdf">Java to Objective-C cheat sheet</a>’ which proved a useful starting point and I was soon at ease. Then, having had a quick look at the language I wanted to get started debugging the iPhone implementation of the SQLite POC.<br /><br />I’d recommend that any Java developer interested in the iPhone platform jump right in - they won’t find the learning curve that steep and in my opinion you can apply all the software development fundamentals that one is often required to know in the Enterprise Java domain to this (much smaller) platform.Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com0tag:blogger.com,1999:blog-3059468813255868046.post-88752827638836172922009-05-02T16:51:00.003+01:002009-05-03T11:08:55.562+01:00Why this is hereAs an experienced Enterprise Java developer forced onto the iPhone development platform, I have noticed many opportunities where the experience I have gained in the enterprise can be used to good effect on even Apple’s smallest public platform.<br /><br />To me at least it seems that Java developers with some good development experience behind them can lend much to this relatively new platform and the community that is rapidly building up around it. I did not find this so much to be the case when working with J2ME which at the time (2004) made it prohibitively expensive in performance terms to stick rigorously to respected patterns and frameworks. The iPhone platform is arguably far more powerful than that J2ME stack I developed for in the past and therefore allows the developer to pay more attention to good design and development practice.<br /><br />In this blog I hope to look at problems encountered on the iPhone platform and how they were solved with lessons learned from J2EE development.Elliot Westhttp://www.blogger.com/profile/07923631905285214423noreply@blogger.com0