April 2015 – Fish Eye View

Quality of Life Reduced Question Set: Bristol Open Data

https://public.tableau.com/javascripts/api/viz_v1.jsThis visualisation operates upon a reduced set of questions from the Quality of Life indicators. This data has been provided by the Bristol City Council under the open data initiative (https://opendata.bristol.gov.uk/).

Using this view the reduced question set can be examined across all the wards as an average of beta for particular question across all wards in Bristol.

Click on a question to focus on it and to examine the beta value across all the wards. A count of wards with positive and negative beta values is also shown. These should correspond to the total green/red marks seen.
The click on a ward to examine the response over time and see the trend line (associated with beta).

Java and Apache Spark used to generate the csv data files.

Link: Dashboard

Criteria for beta calculation: minimum three years data should be available.

Reduced Question Set:

[codesyntax lang=”email”]

% respondents who usually buy fairtrade foods
% people in employment
% respondents book tickets online
Liveability indicator
% respondents who have problem from fly posting
% respondents who are carers 50 plus hours per week
% respondents who have chosen locally grown food to tackle climate change
% respondents who were victims of crime and reported  to the police
% respondents who have used the local tip or household recyling centre
% respondents in receipt of a means tested benefit
% respondents who say there is a problem withdrug dealing in their neighbourhood(includes does not apply/don't know)
% respondents who have been discriminated against or harassed because of ethnicity/race
% respondents who sometimes buy or consume locally grown food
% respondents satisfied with the availability of council/housing association housing
% respondents who are overweight and obese
% who agree that a directly elected mayor is improving the leadership of the city
% respondents who think that the appearance of their area has got better in the last two years
% respondents with problem of cars blocking local pavement
% respondents who find it difficult to get by financially
% respondents who feel locally, antisocial behaviour is a problem
% respondents who think shops have got better in the last two years
% respondents who think schools have got worse in the last two years
% respondents who disagree that the council provides value for money
% respondents who don't have the internet at home
% respondents who are underweight
% respondents who use internet banking
% respondents who agree they can influence decisions that affect public services they use
% respondents unable to use the internet
% respondents who apply for Council or Government services online
% respondents satisfied with the local tip or household recycling centre
% respondents who think drug misuse and drug dealing has got worse in the last two years
% respondents whose combined energy bill per quarter is £300-£399
% respondents satisfied that open public land is kept clear of litter and refuse
% respondents with a problem of abandoned supermarket trolleys in their neighbourhood
% respondents who feel they belong to neighbourhood
% respondents who took 3 or more return long haul flights in the past year
% respondents who are satisfied with the state of repair of their home
% respondents who have a car or van available for use by them or members of the household
% respondents who have volunteered for charity or community every month
% respondents who sometimes buy fairtrade foods
% respondents who say street litter is a problem
% respondents satisfied with their job
% respondents with easy access to a doctor
% respondents satisfied that public land is kept clear of litter and refuse
% economically active respondents who are unemployed and available for work
% respondents who did not take any return long haul flights in the past year
% respondents who have been discriminated against or harassed because of sexual orientation
% respondents who use a public computer
% respondents with parking issues
% respondents satisfied with amount of parks and green spaces
% respondents who live in council or housing association accommodation
% respondents who agree that people take responsibility for their children
% respondents who have been a victims of crime in the last 12 months
% respondents who cook at home using fresh and raw ingredients
% respondents who don't use dental services
% respondents who say graffiti is a problem
% respondents who have volunteered for charity or community at least once a month
% respondents with no educational or technical qualifications
% respondents who have problem from fly tipping
% respondents keen to learn the internet
% respondents who use the internet at work
% respondents who think shops have got worse in the last two years
% respondents who feel safe when outside in their neighbourhood after dark
% respondents who live in rented or tied accommodation
% respondents satisfied with the maintenance of footpaths

% respondents who have been discriminated against or harassed because of religion

% respondents who use the internet at home

% respondents who search the internet

% respondents who say discarded needles and syringes are a problem in their neighbourhood

% respondents who have noise from industrial commercial or construction sites

% respondents who smoke

% respondents who think parks and public spaces has got worse in the last two years
% respondents who think the area they live in will be better in five years time

% who feel police and local public services are successfully dealing with issues of crime and anti-social behaviour in their area
% respondents who are obese
% respondents who have someone use the internet for them
% respondents who think their area is a good place to bring up children
% respondents who use NHS dental services
% of respondents who have access to the internet and use it
% respondents who use digital technology to create content

% respondents who took 2 return long haul flights in the past year

% respondents satisfied with cost and availability of housing
% respondents who have their own garden
% respondents who say traffic congestion is a problem in their neighbourhood
% respondents who have been discriminated against or harassed because of gender
% respondents who shop online
% respondents who usually buy or consume locally grown food
% respondents who agree sexual harassment is an issue in Bristol
% respondents who have been discriminated against or harassed because of age
% respondents who say drug dealing is a problem in their neighbourhood
% respondents who think drunk and rowdy people in public places is a problem
% respondents who think job opportunities has got worse in the last two years
% respondents who use the internet when out and about
% respondents satisfied with jobs in the neighbourhood
% respondents who agree ethnic differences are respected in their neighbourhood
% respondents happy using the internet
% respondents who say drug dealing is a serious problem in their neighbourhood.
% respondents who agree that people treat other people with respect in their neighbourhood
% respondents who eat home cooked 4 times a week
% respondents unemployed
% respondents satisfied with the bus service
% respondents who feel safe when outside their neighbourhood during the day
% respondents satisfied with  local tips / household recycling centres
% respondents who say vandalism is a problem in their neighbourhood
% respondents who say personal safety is a problem in their neighbourhood
% who live in owned private or tied accommodation
% respondents who think drug misuse and drug dealing has got got better in the last two years
% respondents who live in households with a smoker
% respondents who think antisocial behaviour has got got better in the last two years
% respondents who think antisocial behaviour has got worse in the last two years
% respondents who code
% respondents who are dissatisfied with the way the council runs things
% respondents who say they have problem with personal safety in their neighbourhood(includes does not apply/don't know)
% respondents who feel safe indoors after dark
% respondents not interested in using the internet
% respondents who own their own homes
% respondents who eat food grown by themselves or by people they know
% respondents satisfied with academic standards of local schools
% respondents satisfied with markets
% respondents who have access to the internet
% respondents who took 1 return short haul flight in the past year
% respondents who have been discriminated against or harassed because of disability
% respondents who are satisfied with the way the council runs things
% respondents who took 1 return long haul flight in the past year
% respondents who say insensitive development is a problem in their neighbourhood
% respondents with easy access to childcare (adult survey)
% respondents satisfied with leisure facilities/services for children under 12 (adult survey)
% respondents satisfied with libraries
% respondents with easy access to training or education
% respondents who did not take any return short haul flights in the past year
% respondents satisfied with neighbourhood
% respondents who feel dog fouling is a problem in local area
% respondents who took 3 or more return short haul flights in the past year
% respondents who have noise from neighbours
% respondents who feel crime has got worse over last 3 years
% respondents who say neglected or derelict buildings or land is a problem in their neighbourhood
% respondents who think their neighbourhood has got worse over the last 2 years
% respondents who think job opportunities has got got better in the last two years
% respondents who say their neighbourhood is getting better
% respondents with noise from fireworks
% respondents who think the police and council succesfully respond to anti-social behaviour
% respondents who can't afford the internet
% respondents who are willing to call themselves disabled
% respondents who live in households with someone who smokes regularly within the home
% respondents whose combined energy bill per quarter is £400 or more
% respondents satisfied with health services
% respondents who agree people from different backgrounds get on well together
% respondents who don't have a garden or allotment
% respondents buying energy efficient light bulbs
% respondents satisfied with general household waste collection
% respondents who think drug use is a problem in their area

[/codesyntax]
https://public.tableau.com/javascripts/api/viz_v1.js

Javascript: Playing with Prototypes – II

Let us continue the discussion about Prototypes in Javascript and show the different ways in which inheritance can work. Inheritance is very important because whether you are trying to extend the JQuery framework or trying to add custom event sources in Node.JS you will need to extend an existing JS object.

Let us remember the most important mantra in JS – “nearly everything interesting is an object, even functions”

Objects are mutable, primitives (e.g. strings) are NOT!

Let us first introduce the example. There is a base object: Person which has two properties ‘id’ and ‘age’ and getter/setter methods for these. We want to create a child object: Student, which should inherit the id and age properties from Person and add its own read-only ‘student id’ property.

[codesyntax lang=”javascript”]

/*
Base object: Person
*/
function Person(id)
{
  this.id = 0;
  this.age = 0;
}

/*
Add set/get methods for Age and Id
*/
Person.prototype.setId = function(id)
{

  this.id = id;
};

Person.prototype.getId = function()
{
  return this.id;
};

Person.prototype.setAge = function(age)
{

  this.age = age;
};

Person.prototype.getAge = function()
{
  return this.age;
};


/*
Child object Student which should extend properties and methods from Person
*/
function Student(sid)
{
  this.sid = sid;

  /*
  Constructor for Person (to be safe)
  */
  Person.call(this);
  /*
  Student Id getter
  */
  Student.prototype.getSid = function()
  {
    return this.sid;
  }
}

[/codesyntax]

There are different ways (patterns) of implementing ‘inheritance’ based (Inheritance Methods):

Pattern 1: Student.prototype = Object.create(Student);
Pattern 2: Student.prototype = Object.create(Person.prototype);
Pattern 3: Student.prototype = new Person;

Below is the snippet of code we use to probe what happens in each of the three cases. Two instances of Student are created (s1 and s2). Then we examine the prototypes and assign values to some of the properties.

[codesyntax lang=”javascript”]

<Inheritance Method: one of the three options above>

var s1 = new Student(101);
var s2 = new Student(102);

console.log("S1",s1);
console.log("S2",s2);
console.log("Proto S1",Object.getPrototypeOf(s1));
console.log("Proto S2",Object.getPrototypeOf(s2));
if (Object.getPrototypeOf(s1) == Object.getPrototypeOf(s2)) {
  console.log("Compare prototypes:",true);
}

console.log("\n\n");

s1.setId(1);
s1.setAge(30);
console.log("S1",s1.getAge());

s2.setId(2);

console.log("Compare Id S1:S2",s1.getId(),s2.getId());

s2.setAge(20);
console.log("S2 set age 20");

console.log("S1 age",s1.getAge());
console.log("S2 age",s2.getAge());

[/codesyntax]

Let us look at what happens in each case:

1) Student.prototype = Object.create(Student);

Output:

[codesyntax lang=”php”]

S1: { sid: 101, id: 0, age: 0 }
S2: { sid: 102, id: 0, age: 0 }
Proto S1: { getSid: [Function] }
Proto S2: { getSid: [Function] }
Compare prototypes: true


/Users/azaharmachwe/node_code/NodeTest/thisTest.js:73
s1.setId(1);
^
TypeError: Object object has no method 'setId'
at Object.<anonymous> (/Users/azaharmachwe/node_code/NodeTest/thisTest.js:73:4)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)
at startup (node.js:119:16)
at node.js:901:3

[/codesyntax]

The surprising result is that an exception is thrown. It seems there is no method ‘setId’ on the Student instance. This means that inheritance did not work. We can confirm this by looking at the prototype of S1 and S2 instance. Only the getter for student id defined in the Student object is present. We have not inherited any of the methods from Person.

But if we look at the list of attributes we see ‘id’ and ‘age’ present. So it seems the attributes were acquired somehow.

If we look at the way we define the Person object we actually add the ‘id’ and ‘age’ attributes to the instance (i.e. we use this.id) where as the accessor methods are added on the prototype. When we create an instance of Student as Student.prototype = Object.create(Student) we correctly set the attributes as they are defined at the instance level.

If the line in bold is removed then you will only see the Student level attribute (‘sid’).

2) Student.prototype = Object.create(Person.prototype);

Output:

[codesyntax lang=”php”]

S1: { sid: 101, id: 0, age: 0 }
S2: { sid: 102, id: 0, age: 0 }
Proto S1: { getSid: [Function] }
Proto S2: { getSid: [Function] }
Compare prototypes: true



S1 30
Compare Id S1:S2 1 2
S2 set age 20
S1 age 30
S2 age 20

[/codesyntax]

No errors this time.

So we see both S1 and S2 instances have the correct attributes (Person + Student) prototypes for both contain the getter defined in Student and both have the same prototype. Something more interesting is the fact that we can set ‘age’ and ‘id’ on them as well showing us that the attribute setters/getters have been inherited from Person.

But why can’t we see the get/set methods for ‘age’ and ‘id’ on the Student prototype? The reason is that with the call to Object.create with the Person.prototype parameter we chain the prototype of Person with that of Student. To see the get/set methods for ‘age’ and ‘id’ that the Student instance is using add the following line to the probe commands:

console.log(“>>”,Student.prototype.__proto__);

This proves that the object is inheriting these methods at the prototype level and not at the object level. This is the recommended pattern for inheritance.

3) Student.prototype = new Person;

This is a method you may see in some examples out there. But this is not the recommended style. The reason is that in this case you are linking the prototype of Student with an instance of Person. Therefore you get all the instance variables of the super-type included in the sub-type.

Output:

[codesyntax lang=”php”]

S1: { sid: 101 }
S2: { sid: 102 }
Proto S1: { id: 0, age: 0, getSid: [Function] }
Proto S2: { id: 0, age: 0, getSid: [Function] }
Compare prototypes: true



S1 30
Compare Id S1:S2 1 2
S2 set age 20
S1 age 30
S2 age 20

[/codesyntax]

Note the presence of ‘id’ and ‘age’ attributes with default values in the prototypes of S1 and S2. If the attributes are array or object type (instead of a primitive type as in this case), we can get all kinds of weird, difficult to debug behaviours. This is the case with frameworks where a base object needs to be extended to add custom functionality. I came across this issue while trying to create a custom Node.JS event source.

Wrong way to extend: A Node.JS example

I have seen many Node.JS custom event emitter examples that use pattern number (3). The correct pattern to use is pattern (2). Let us see why.

The code below extends the Node.JS EventEmitter (in ‘events’ module) to create a custom event emitter. Then two instance of this custom event emitter are created. Different event handling callback functions for the two instances are also defined. This will allow us to clearly identify which instance handled the event.

In the end we cause the custom event to fire on both the instances.

[codesyntax lang=”javascript”]

var ev = require("events");

/*
Create a custom event emitter by extending the Node.JS event emitter
*/
function myeventemitter(id)
{
  this.id = id;
  ev.EventEmitter.call(this);
}
/*
Try different ways of extending
*/

myeventemitter.prototype = new ev.EventEmitter;

myeventemitter.prototype.fire = function()
{
  console.log('\nFire',this.id);
  this.emit('go',this.id);
}

/*
Initialise two instances of the custom event emitter
*/
var myee1 = new myeventemitter("A");
var myee2 = new myeventemitter("B");

/*
Define callbacks on the custom event ('go')
*/
myee1.on('go',function(id)
{
  console.log("My EE1: Go event received from",id);
});

myee2.on('go',function(id)
{
  console.log("My EE2: Go event received from",id);
});

/*
Cause the custom event to fire on both the custom event emitters
*/
myee1.fire();
myee2.fire();

/*
Dump the prototype of our custom event emitter
*/
console.log(myeventemitter.prototype);

[/codesyntax]

Note we are using pattern (3) to extend the EventEmitter:

myeventemitter.prototype = new ev.EventEmitter;

We expect that custom events fired on instance 1 will result in the event handling function on instance 1 being called. The same thing should happen for instance 2. Let us look at the actual output:

[codesyntax lang=”javascript”]

Fire A
My EE1: Go event received from A
My EE2: Go event received from A

Fire B
My EE1: Go event received from B
My EE2: Go event received from B
{ domain: null,
_events: { go: [ [Function], [Function] ] },
_maxListeners: 10,
fire: [Function] }

[/codesyntax]

This looks wrong! When we cause instance 1 to fire its custom event it actually triggers the event handling functions in both the instances! Same happens when we try with instance 2.

The reason as you may have already guessed is that when we use pattern (3) we actually attach the JSON object that holds the individual event handling functions to the prototype (variable name: _events). This can be seen in the above output.

Therefore both instances of the custom event emitter will have the same set of event handling functions registered because there is only one such set.

To correct this just switch the extension patter to (2):

[codesyntax lang=”javascript”]

Fire A
My EE1: Go event received from A

Fire B
My EE2: Go event received from B

{ fire: [Function] }

[/codesyntax]

The output now looks correct. Only the instance specific callback function is called and the prototype does not store the event handling functions. Therefore each instance of the custom event emitter has its own set for storing event handling functions.

Bristol Government: Open Data Initiative

Bristol City Council (BCC) is now publishing some of their data sets online as part of the Open Data initiative.
This is a VERY positive move and I too hope that this leads to the development of ‘new’ solutions to the city’s problems.
More information can be found here: https://opendata.bristol.gov.uk

The Tableau Viz below uses the Quality of Life Indicators data from across Bristol. This is available from the BCC website. The data set has a set of questions (about 540) asked across the different wards in Bristol (about 35) on a yearly basis starting from 2005 till 2013. Obviously data is not available across all the dimensions, for example the question:
“% respondents who travel for shopping by bus” for the Redland ward is available only from 2006-2010.

The raw data from the Open Data website was processed using Apache Spark’s Java Libraries. This was then dumped into a data file which was imported into Tableau.

Link: Dashboard

The heat map below plots the regression slope of the survey results over the years (beta) against the Questions and Wards.
Criteria for beta calculation: minimum three years data should be available.

https://public.tableau.com/javascripts/api/viz_v1.js