Sunday, 5 October 2014

Secret Scripts: the antipattern

The Secret Script is something I've been aware of for a while and, although I'm fairly sure I didn't make it up, I can't find any reference to it on Google. So I'll need to explain what I mean rather than just supplying a link.

A Secret Script is some procedure, possibly manual or automatic that you have to know about to complete a software build. It isn't intended to be secret as such, in fact it is one of those things that everyone knows, except the new guy who can't get his build to run.

You can see how these things start, especially in a small team under pressure to deliver. Cut a quick ant script to create a directory and copy some files to it, hack the maven settings file, and make sure you set up that environment variable. Sure, now everything works, and it works every time, so we can forget about it.

If you are building open source projects, where you put the entire thing up in the cloud with the expectation that anyone in the world can pull it down and build it on their machine you cannot have secret scripts. Not ever. If I pull down an open source project and it doesn't build first time that's a signal to delete it and find something else. But closed source have more leeway here, especially if the team is small and information can be easily passed around.

Note that I am not referring to documented steps. You can have a couple of dozen manual steps in your ReadMe file and, as long as that forms part of the project, ie anyone pulling down the software gets the ReadMe file in an easy-to-find directory. Those steps aren't secret, they are easily discoverable.

I said closed source projects have more leeway but not very much more. Every new hire programmer needs to get past these secret scripts. You either have someone who knows sit with them to get them going, which is time consuming, or you leave them to figure it out for themselves, which is even more time consuming, not to mention discouraging for the new guy. There is always the danger than the people who know have forgotten they had to edit some config file two years ago anyway and will be just as baffled as the new guy as to why his build fails.

And with modern tools there should be no need for this. If you're a Java shop you are probably using maven or maybe ant. On a new machine anyone ought to be able to type mvn or ant on a command line and get a working build. If they can't the system is broken with secret scripts.

Secret scripts have a sort of mirror pattern which is, unfortunately, still an antipattern. I call these decoy scripts. A decoy script is a script or procedure that is right there in plain sight looking like it is important and useful, but which does not actually work and everyone but the new guy knows to ignore it. The new guy tries it out, finds it doesn't work, and tries to fix what is obviously a local problem. The certainty that everyone else has this script working will lead him to waste hours trying to fix what has to be a problem local to his machine.

Here are some examples of decoy scripts:
  • Out of date instructions in the ReadMe file.
  • Unit tests that don't pass (and aren't flagged as @Ignore)
  • old ant scripts that refer to invalid paths
These things are fairly easily avoided, and fixing them can save a lot of time.

Saturday, 4 October 2014

My trousers caught fire!

There's the proof. Trousers and socks ablaze. Fortunately I was wearing other trousers at the time.

This time of year we burn the piles of prunings and other tree trimmings we've accumulated since the start of winter (it is spring here in NZ). We'd started it with some paper in a cardboard box which I took to the back door afterwards to fetch some tools.

When the blaze was starting to die down we headed back and found the box was just ashes and my gardening trousers that lay beside them were alight. Obviously the box had been too near that fire. Easily fixed and no damage done, except to a very old pair of trousers.

Could have been worse though.

Wednesday, 1 October 2014

Domestic Dramas

We got back from hols and, as you do, turned on the espresso machine. Bang!
What? Turned it off then back on. Nothing. But not on either. Checked the fuse box. Yep, one of the cutouts has flipped. I took the back off the espresso machine and wiggled some wires, one of which looked a bit blackened. Ficked the cutout back on and tried again. Bang! There were visible sparks this time. Okay, that's not good.

I disconnected the espresso machine altogether, called a service shop and arranged to take it in there to get fixed. It's a wonderful machine, one of those Electras with a brass eagle on the top, and it does a good job, been doing it for nearly 20 years now.

Meanwhile I fetched the emergency backup espresso machine (yes, we do have an emergency espresso machine cos of a more minor problem 3 years ago) and set it up on the kitchen bench. Espressos followed.

Getting the main machine to the repair shop is a little tricky because it has a plumbed in water supply. I remember when we set this up and there's a shut off on the incoming pipe so the machine can be disconnected without getting water everywhere. It is hidden behind a panel in the bench below. I took off the panel, found the shut off and turned it off. Then I set to disconnecting the machine.
The moment I got the water supply line off the machine I got water sprayed in my face and pretty much everywhere. What? I shut off the water! I did! (this, among other things was what I was yelling to Mrs as I asked her to go shut off the mains).

That's complicated too. We gather water from our roof and store it in a tank, then a pump sends it to the house. The pump is smart, it detects the drop in pressure when we turn on a tap and cranks up to maintain it. So Mrs ran outside to the pump switch while I jammed my finger across the tube to stop it squirting. It seemed to take a long time.

Next step was to find out what was going on with the shut off that didn't. I got a wrench and took it off... and noticed water was still dribbling steadily from the pipe. Huh? We turned off the pump. The kitchen tap is still on and is not running, that proves it.

Except that this particular pipe end is lower than all the taps because it under the bench. That means it is below the top of the water tank, so there's some pressure even without the pump running. I need to turn off the tap on the tank. Leaving Mrs in charge of a bucket to catch the dribble I went out and struggled with the tap which hasn't been turned in at least 5 years and didn't feel inclined to now until I started hitting it. That sorted out the dribble. So we were no longer in danger of flooding the kitchen any more than it was already.
I had enough bits and pieces to contrive a bung in the pipe and then I could turn the water on again. It held okay. Whew.

That's enough excitement for now, but I am going to take a long hard look at that shut off that didn't before I put this back together.

Sunday, 14 September 2014

Setting up Drools Execution Server

The Drools Execution server (6.2.0-Beta2) has a demo video which looks good, but I needed to know how to set it up on my own machine.

It is not quite obvious how to do this (but this is beta code, so there is a good excuse for that). Still, I thought I'd document how I did it here to save others time.

I'm running Xubuntu 14.04 but the operating system is unlikely to be relevant. Translate as necessary to Windows or OSX path names.

First you need an application server. I'm using Wildfly 8 (used to be called JBoss). Get Wildfly-8.1.0.Final.zip and unzip it into a directory we will call . You need to add a couple of user names before you start it up. Use the /bin/add-user.sh script to add two users we'll call user1 and user2. Make user1 an admin user and user2 an application user. Leave the groups field empty for user1 and (confusingly) make the group of user2 'admin'.
For the question after groups enter 'no'.

Now you can start Wildfly using /bin/standalone.sh and browse to http://127.0.0.1:9990 and log in as user1.

Download the two war files you'll need.
Drools Workbench (this is the Wildfly-specific version)
Drools Execution Server

Edit: I see those two links have gone dead. Here is how to find the right files. Go to this link (it may take a while to load). Now find the JBoss Releases repository. In the lower panel find the org/kie/kie-drools-wb and org/kie/kie-server-services. Under each of those are various versions you can chose from. You want two war files with the same version, bearing in mind the workbench has a specific war file for the app server in each version.

Use the Wildfly Management to load and enable the two war files.
Edit: it is a good idea to shorten the names when you load the war files, that's the second  question during the Wildfly load, and it means your url will be shorter.

We're nearly done...

Log into this URL using user2
http://127.0.0.1:8080/kie-drools-wb-distribution-wars-6.2.0-20140912.025916-106-wildfly8

(or, if you shortened the name you might use http://127.0.0.1:8080/kie-drools-wb)

Okay, now you are in the demo. The ones in the video have some different files loaded but the ones you have now seem to work. You can certainly deploy them and you can certainly call them with a REST client.



Saturday, 6 September 2014

People who say 'Well I think..."

Here I am listening to a radio broadcast of a panel discussion between several
political candidates in our impending general election. It's a well run discussion (thanks to the skills of Wallace Chapman), and unlike some similar events, everyone is behaving well so we all get to learn a lot about who thinks what about what. This is good. Here is the relevant

But what I'm more interested in here is not so much the specifics as the pattern they take. For example there is a question about what the voting age should be. Currently it is 18. I remember (just) when it was 21. Should it be higher? Should it be lower? What do you think?

This seems a good question, certainly worth asking. At 18 we can sign contracts, marry (actually we can do that at 16), vote, buy alcohol and join the army. We don't have any need to conscript soldiers just now, but when we have had such a need it has been tied to the voting age on the understanding that if you're not old enough to vote you're not old enough to be made to risk your life for your country.

So the panel tossed the question back and forth with answers starting with 'well I think...' and speculating that some younger people are old enough to vote and perhaps they should. Then others think it is about right or maybe could be higher, especially with alcohol purchasing.

What no one suggested is actually studying the people involved. Do we have any good data on how well informed 10 year olds actually are? 12 year olds? 14? 16? 18? Surely this is critical to setting the age. We know from other studies that IQ has been shifting upwards for years. This is called the Flynn Effect (Flynn is a New Zealand scientist, but the effect is observed world wide). So it is not unreasonable to suppose that our understanding about how younger people  make decisions is out of date.

Step back a bit, though. This is really not about the specifics. It is about gathering data. It is not enough to base policy on 'well I think...', and it seems we do too much of this. What any of us 'think' about some issue is far less relevant than what the data says.

Another example just to round it out: we have a minimum wage in NZ. Some people think it should be raised, others want to scrap it. The latter tend to say that raising it will cost jobs. This is a 'well I think...' statement. It is very easy to show data that raising the minimum wage does not cost jobs by looking at what has happened when this was done elsewhere. I've yet to see a study referenced that shows it has cost jobs. Maybe there is one, but let's see it instead of having people say 'well I think...'

So watch out for people saying 'well I think...'. If you can, ask them why they think that. Press them for data, some reference to some properly conducted study that demonstrates what they are saying is true (or at least not false, which is subtly different but okay). If you want to pursue this further get hold of Karl Popper's The Open Society and its Enemies which is about some other things as well but he argues better than I can for evidence based policy. Popper wrote this while he was lecturing in Christchurch, NZ during WWII (as an Austrian Jew he needed to leave Europe, but his German heritage made him unwelcome in the armed forces on our side). So there is a local connection there too.

Saturday, 2 August 2014

Do we really want one law for all?

One Law for All. It's a good slogan and, like Motherhood and apple pie, pretty difficult to object to. Some of our politicians have started saying this. We have an election coming up here in New Zealand. The sub text of the slogan is that some people feel that Maori are getting a better-than-fair deal. Specifically two things:

We have a separate Maori electoral role which elects its own MPs. Sounds very apartheid at first glance, but the first difference is that Maori themselves decide if they vote on the General role with the rest of us, or on the Maori role. And when I say 'decide' I mean each individual makes that choice, it isn't delegated to some representative body that is out of their control. If Maori don't want the role they can just (individually) stop using it. The other difference is that the MPs are real MPs, with as much status as any other. So not apartheid.

The second thing is that there is consultation about various issues such as how land gets used, where motorways get built and so on. One of the objections to this is that such consultation takes up time and time is money.
These two privileges get some people fired up and using the word 'unfair' and so on. I guess I'm in the demographic that gets fired up: white, middle class, middle aged, male. So why aren't I?

The answer is, quite simply, history.

If you were on the first boat load of people to arrive in this country and you were setting up a society for the first time, from scratch, 'One Law for All' is just obvious. But we aren't. Not by a long shot.

When the English took over New Zealand they did not do it by conquest. They signed a treaty with Maori. There are arguments over what that treaty says, not uncommon with legal documents, but it is pretty clear that there was a promise that Maori would be able to keep the rights they already had over the land. This is more complex than it might sound. In the UK the land is owned by 'the crown' and people actually own a 'title' to the land, which is a licence to use it rather than ownership. I've heard there are strange exceptions to that but it is generally the case. It is the same here in New Zealand. So the New Zealand land effectively became owned by Queen Victoria in 1940, with Maori now having a title.

This allows the state to exercise compulsory purchase when, say, they want to build a motorway through your house, dig for oil under and fly planes over it. We don't have a simple ownership right to our land, we have a title.

Okay, almost before the ink on the treaty was dry boatloads of settlers began arriving from England and they wanted land. The history is a bit complicated but dirty deals were done, fighting broke out, troops from the UK were brought in to quell the rebellious natives and land was confiscated. It wasn't only confiscated, there were other ways of prising Maori apart from their land which were more subtle. Remember the compulsory purchase? It was decided that Maori land did not need to be 'purchased' as such, it could be just taken if the state needed it. Since it was simpler to take Maori land rather than pay money for other land you can guess that was the preference. While the fighting was over fairly quickly, the compulsory 'purchase' arrangements continued well into last century.

Less formal was the casual prejudice that sidelined Maori economically. I grew up on a farm that was confiscated land and we had a Maori farm hand. I'm not sure my father and the farm hand gave much thought to the history, or even if they knew it. But the farm hand wanted to buy a car and he needed a loan. My father knew the bank manager well so he talked to my father about it who approached the bank manager on his behalf. My father told me he explained to the manager that this farm hand was in steady work, was a reliable character and that he himself was prepared to guarantee the loan. The bank manager explained that this, then state owned, bank did not lend to Maori. So there was no loan. My father had a loan, but he was white.

The result was that all the best land got into the hands of white settlers who cleared it and farmed it and built our economy on it.

But the unfairness of this came back to haunt us eventually and late last century we started doing something about it. The first major compensation settlement was for Tainui, the people who used to own the rich Waikato area. They got some land back and cash. I remember at the time people pointing out that Tainui were magnanimous. The value of the compensation was nothing like what they had lost. Everyone understood that the only land that could be returned was land that had not passed into private hands, and there was no way the cash could make up for it. This has been the pattern for subsequent settlements. No one, at that point, suggested there ought to be one law for all. One law for all would see farmers turned off their lands and any government trying to make that happen would never survive another election.

Since then there have been moves to respond to this magnanimity. The Maori electorate seats were already there, they've been in place since 1867, shortly after the fighting and they were brought in to ensure Maori could vote. In recent years the number has been expanded from 4 to 7, to more fairly reflect the number of people on the Maori electoral roll. Consultation with the local iwi (tribe) over various issues is commonplace, though not always comprehensive or even popular, but it is a move in the right direction.

We now get situations like this where a taniwha, or water monster, held up a highway construction for, it was claimed, 3 weeks while local Maori were consulted. Now, some people got very, very annoyed about this and talked up the cost of the project, the amount of time wasted and the 'ridiculous belief' in taniwha. I saw it as real progress. We often accomodate various 'ridiculous beliefs' such as the value of heritage and religious sites and surely Maori, to whom we owe so much, ought to get similar accomodation.

But let's not get carried away about having one law for all. If we insist on that it will cost us dearly.

Tuesday, 29 July 2014

Connectivity

I live in the woods, so getting the internet here is not as obvious as it is to most of you. I few months back when I realised my neighbour had wired broadband I tried getting it to my house too. The technician came out in a van and a hi-viz vest, poked a the phone line connection to the house, made that ticking sound with his tongue and drove off. Soon after I got a letter saying I was too far from the exchange to support wired broadband, oh and please send back the modem they had couriered me.

So, excitement over, I continued using my phone as a wifi hotspot, sharing its 3G connection. That works quite well but my plan is capped at 3GB. When I say capped that means no more data after that. Some plans allow you to pay something and put a bit more data on. Not this one. Capped means capped. It is actually an old plan and, based on the odd letter and phone call I get from Vodafone they would like me to switch, but the only things they have to switch to have less data and more voice time. Since I almost always hit the end of my data and never run out of voice this is not attractive.

You may imagine I look at the wired broadband plans that offer 80GB caps (they're pretty well always capped in NZ) with envy. I can add to my 3GB by going to the library where I get 200MB a day, but I feel kind of 3rd world doing that. My favourite cafes offer unlimited but if I camp there all day I end up drinking too much coffee, though I tend to get all my updates done by making sure I breakfast out most weeks.

But when I get doing something like installing new software, or developing new software (which inevitably needs me to pull down some new libraries) I can easily find my quota is blown. Checking G+ gobbles up about 50MB so I don't do that every day.

Mrs has a T-stick, a little USB 3G modem she plugs into her laptop. It will actually plug into a wireless modem we have so it would be a lot like everyone else has, ie wifi all over the house connected to broadband. Her plan has a cap as well, an extendable cap. That's because it is with Telecom instead of Vodafone. So I got thinking: if I got another T-stick I'd get another 3GB. Extending Mrs for another 3G is more expensive than a separate one, besides she doesn't quite trust me not to chew through all her quota (and she is right not to).

These T-sticks come trumpeting their support for Windows and Mac machines... but not Linux. So I expected to have to scratch around on the web and find out what to do. Mrs' T-stick doesn't work OOTB, I tried it. I believe it is possible and when I convince her to switch to Linux I'll need to figure that out. But my new T-stick (pictured... it's not a tampon) just worked. No config, nothing at all. It shows up as a wired connection. For anyone else trying this it is the E3531 T-stick and I have it running on Ubuntu 13.10. It has a little web server in it you can browse to if you need to change the config or send/receive TXT, or you can ignore it. Just browse to http://192.168.8.1

I'm not actually in the woods right now, I'm down in the city (Auckland) so I have it on my laptop, but when I get home I will plug it into the modem and use the house wifi. I probably need about 4GB/month so I can either top it up when I need to or I can use my phone, though I expect I will change the plan on the phone now, quite possibly I'll switch it over to Telecom since they seem to do a better job.

There's a second reason to switch the phone to Telecom. We don't have a land line connected. That's our choice, we used to have one, but we never used it because we have the mobiles. So when we had a big storm a while back the Vodafone network went down. The Telecom one stayed up. It meant Mrs could get online but with both phones on Vodafone we could not voice-call (and I had no internet). We didn't need to call anyone, but it might have been handy if we had an emergency. So it seems prudent to spread it around, especially since it is no longer vital to keep that 3GB plan.

With the house wifi providing internet I can use it like the rest of you do, ie multiple devices connecting to it, hang a USB drive off the modem for simple file exchange and backup and make better use of our wifi printer. My phone needs to be plugged into power to hotspot because that chews through the battery, so doing that less will be nice.