Develop Web Application with Node.js, Express.js REST Service and Neo4j (neo4j-js module) – Updated



This article has been updated with Express 4. Click here to see the updated article.


This article is a proof of concept towards make a web application with Javascript based Web Server with Node.js and Neo4j.

To make this possible, we have selected –

1> Node.js for server side development

2> Rest based web service creation with express.js

3> Graph Database – Neo4j

4> Node.js Neo4j Module Extention (neo4j-js)

We have created a Proof of Concept with Javascript based web server, where we have focused on dealing with Neo4j(Graph Database) with javascript based framework Node.js.

We have taken help of the Neo4j-js module, which are discussed in Neo4j Site and the Github project contribution from Bret Copeland.

Our initial code can be downloaded here.

We have taken reference Data Model of Neo4j Graph Database for this application from our previous knowledge on Neo4j.

For reference of the Neo4j Graph Database Model , take a look at our previous 2 articles –

1> Case Study in Graph Database in Context of Neo4j Cypher Query

2> Neo4j Cypher Queries for Sample Graph Database

Architecture at a glance –

NodeJS Neo4j Architechture

So here are the steps –

Installation –

A> Download and install Node.js from here.

B> To Develop the application we need to install Neo4j-js module for Node.js

Command – npm install neo4j-js (should be connected to internet)

C> We need to install express.js for node.js

Command – npm install express  (should be connected to internet)

D> We need to install http status

Command – npm install http-status (should be connected to internet)

Configuration Code –

Now, we will try to describe the code portion –

 
var application_root = __dirname,
    express = require("express"),
    path = require("path"),
    HTTPStatus = require('http-status'); // To return http status for error codes

var config = require('./config');

Here we have initialised the express.js within javascript variables in respect of Node.js concept.

We have introduced another js file – config.js to put all the neo4j configuration related information.

 
var app = express();

Here we have initialised the express web server in app variable.

 
var neo4j = require('neo4j-js');

Here we have made the Node.js Neo4j-js module available .

 
var neo4jurl = process.env.NEO4J_URL ;
neo4jurl = neo4jurl +'/db/data/';

Here the Neo4j Database server url is set from configuration file.

 
var query = [ 'START me=node:node_auto_index(name={inputusername}) MATCH me<--friend<--friend_of_friend where 
(friend_of_friend.entitytype={inputentitytype}) RETURN friend_of_friend;'	];
var insertquery = [ 'CREATE (user {entitytype:{inputentitytype}, name : {inputname}}) return user;' ];
var queryforallrelation = [ 'start n = node:node_auto_index(name={inputusername}) match(n)--(x)  return x;'];

Here all the query variables are set with Neo4j Cypher Queries.

 
// Express Config

app.configure(function () {
  app.use(express.bodyParser());
  app.use(express.methodOverride());
  app.use(app.router);
  app.use(express.static(path.join(application_root, "public")));
  app.use(express.errorHandler({ dumpExceptions: true, showStack: true }));
});

Here we have made the configuration related to express.js

Rest Services Code –

 
app.get('/api', function (req, res) {
  res.send('Our Sample API is up...');
});

Here we have made our first REST based web service and tested whether the express.js is up.

Our sample api will be  – http://127.0.0.1:1212/api (Get Method)

 
app.get('/friendoffriend/:username', function (req, res){
	res.set({'Content-Type': 'text/json'}); // setting content type
	username = req.params.username;
	type = 'user';
	neo4j.connect(neo4jurl, function (err, graph) { //Connecting neo4J

			graph.query(query.join('\n'), {inputusername : username, inputentitytype :type} ,
                             function (err, results) {	// Query to Neo4j Database through Neo4j-js Module
				if (err) {
					res.send(HTTPStatus.INTERNAL_SERVER_ERROR,'Internal Server Error'); 
                                        // Return Http Status for Error
				}
				else {
					res.send(HTTPStatus.OK,JSON.stringify(results)); 
                                        // Return the json in response
				}
			     });	
	             });
               });

Here we have created REST api to get user information from Neo4j Nodes Collection and so have done through Neo4j-js Extention Library. We have made a query for ‘Friend of Friend’ in Cypher Query Language.

More information about Neo4j Cypher query can be found in Neo4j Cypher Queries for Sample Graph Database.

Our sample api will be  – http://127.0.0.1:1212/friendoffriend/andrew (Get Method)

 
app.get('/insertuser/:username', function (req, res){
	res.set({'Content-Type': 'text/json'});
	username = req.params.username;
	type = 'user';
	neo4j.connect(neo4jurl, function (err, graph) { // Connection to neo4J

			graph.query(insertquery.join('\n'), {inputname : username, inputentitytype :type} 
                             ,function (err, results) {	// Query to Neo4j through neo4j-js module
				if (err) {
					res.send(HTTPStatus.INTERNAL_SERVER_ERROR,'Internal Server Error');
                                        // Return Http Status for Error
				}
				else {
					res.send(HTTPStatus.OK,JSON.stringify(results));
                                        // Return the json in response 
				}
			    });	
	               });
               });

Here we have inserted a node in database through cypher query via REST web service calling.

Our sample api will be  – http://127.0.0.1:1212/insertuser/anotheruser (Get Method)


//Cypher Query with Javascript Callback Example

function neo4jQuery(neo4jurl, query, parameters, giveresults) { 
        //Function with query, parameter and callback as arguments
	neo4j.connect(neo4jurl, function (err, graph) {  // Connection to neo4J
		graph.query(query.join('\n'), {inputusername : 'andrew'} ,
                        function (err, results) { // Query to Neo4j through neo4j-js module

			if (err) {
				giveresults(HTTPStatus.INTERNAL_SERVER_ERROR);  // Return Http Status in callback
			}
			else {
				giveresults(JSON.stringify(results));  // Return the json in callback
			}
		});	
	});
}

//Below is a separation of REST related code and neo4j related code via a javascript callback function
app.get('/allrelations/:username', function (req, res){
	res.set({'Content-Type': 'text/json'});
	username = req.params.username;
	parameters = {inputusername : username};
	neo4jQuery(neo4jurl, queryforallrelation, parameters, function(results){ // the previous function is called
		res.send(results);
	});
});

Our sample api will be  – http://127.0.0.1:1212/allrelations/andrew (Get Method)

Above we have taken another  Query Example from Node.js with Query, Parameter and Callback function.

 
// Launch server
app.listen(1212);

We have made the server to listen at 1212 port.

To run the application, we should use – node neo4jtest.js from command shell.

Update : We have updated  the code-base as per advice from Michael Hunger about improvement of the Post.

Evaluate the coding. Comments are welcome.

Enter your email address:

Delivered by FeedBurner

Quick Glance at Neo4j Cypher Queries for Sample Graph Database


We have tried to show different Neo4j Cypher Queries to show different capabilities of graph database.

To start with graph database preliminary concepts, read Preliminary Concepts of Graph Database.

.We have given below the cypher queries, which are based on a model – which is described in Case Study in Graph Database in Context of Neo4j Cypher Query.

Installation –

Install Neo4J from http://www.neo4j.org/download

Starting Neo4j Server in Windows –

neo4j.bat start

Then go to browser at the address – http://localhost:7474

Now the cypher queries, which we have tested on the Data Console which is in relation to the mentioned model.

Creating Nodes –

CREATE (user {entitytype:’user’, name : ‘bill’}) return user;

CREATE (company {entitytype:’company’, name : ‘CTS’}) return company;

CREATE (project {entitytype:’project’, name : ‘ABC Insurance’}) return project;

CREATE (skill {entitytype:’skill’, name : ‘HTML, CSS’}) return skill;

Some of the Nodes creation examples are shown here.

Creating Relationships –

START user=node(1), company=node(9) CREATE relation = user-[:WORKS_AT]->company return relation;

START user=node(7), company=node(9) CREATE relation = user-[:FOLLOWS]->company return relation;

START user=node(5), skill=node(12) CREATE relation = user-[:HAS_SKILL]->skill return relation;

START user=node(6), seconduser=node(7) CREATE relation = user-[:FRIEND]->seconduser return relation;

START user=node(6), work=node(11) CREATE relation = user-[:CURRENT_ENGADGEMENT]->work return relation;

START user=node(6), work=node(11) CREATE relation = user-[:CURRENT_ENGADGEMENT]->work return relation;

START company=node(9), project=node(11) CREATE relation = company-[:EXECUTING_UNDER]->project return relation;

Some of the relation creation examples are shown here.

Removing node –

START nullnode=node(0), DELETE nullnode; // Delete a Node

Fetching Nodes and Relationships –

start n = node(*) return n;// See all the nodes
start r = relationship(*) return r; //see all the relationships

start n = node:node_auto_index(name = ‘john’) return n;// Selecting particular user –>Selecting Data by Indexes

START n=node(*) WHERE has(n.name) RETURN n; // Selecting those nodes where we have name properties

START n=node(*) WHERE has(n.name) and (n.entitytype = ‘skill’) RETURN n; // Selecting nodes where entitytype = ‘Skill’

index –indexes // To Get the Indexes in nodes and relationships

START n=node(*) WHERE has(n.name) and n.name =~ ‘j.*’ return n; //Selecting nodes from indexes where node name starts with “j”

START n=node(*) WHERE has(n.name) and n.name =~ ‘.*j.*’ return n; //Selecting nodes from indexes where node name contains “j”

start n = node(6) match(n)–(x) return x; //return all the related nodes with node 6

start n = node:node_auto_index(name=’andrew’) match(n)–(x) return x; //return all the related nodes with node name = ‘andrew’

start n = node:node_auto_index(name=’andrew’) match(n)<–(x) return x; // return all nodes which are having inbound relationship with ‘andrew’

start n = node:node_auto_index(name=’andrew’) match(n)–>(x) return x; // return all nodes which are having outbound relationship with ‘andrew’

// Note : Here all entitytype (skill, project, user, company) to which andrew is having outbound relationship will return

START n=node:node_auto_index(name=’andrew’) MATCH (n)-[:WORKS_AT]->(x) RETURN x;

//Match by relationship type and outbound nodes

START n=node:node_auto_index(name=’linda’) MATCH (n)-[:FRIEND]->(x) RETURN x;

// Here result will be ‘andrew’ who is outbound relationship to ‘linda’

START n=node:node_auto_index(name=’linda’) MATCH (n)<-[:FRIEND]-(x) RETURN x;

// Here result will be ‘john’ for whom ‘linda’ is outbound relationship

START n=node:node_auto_index(name=’andrew’) MATCH (n)-[r:WORKS_AT]->() return r;

// Match by relationship type and use an identifier

START a=node:node_auto_index(name=’andrew’) MATCH (a)-[:WORKS_AT]->(b)<-[:WORKS_AT]-(c) return a,b,c

//return Multiple relationships

// Here all the people, who are working in ‘IBM’ will be returned, as ‘andrew’ is working in ‘IBM’

START n=node(*) WHERE has(n.name) and (n.name =~ ‘john.*’) RETURN n;

// Where condition in the Query and ‘property exists in node’ clause and Like clause

START n=node(*) WHERE n.entitytype? = “skill” RETURN n;

// Default true for nodes if property is missing and all other matching nodes

START n=node(*) WHERE n.entitytype! = “skill” RETURN n;

// Only for nodes with matching nodes.  Note : “!=” and “! =” are not same

START persons=node(*), company=node:node_auto_index(name=’IBM’) WHERE has(persons.name) and (persons.entitytype=’user’) and not(persons–>company) RETURN persons;

// Those persons will be returned who are not related to ‘IBM’riend

START persons=node(*), company=node:node_auto_index(name=’IBM’) WHERE has(persons.name) and (persons.entitytype=’user’) and (persons–>company) RETURN persons;

// Those persons will be returned who are has relation to ‘IBM’ (here note the ‘WORKS_AT’ and ‘FOLLOWS’ relationship)

START persons=node(*), company=node:node_auto_index(name=’IBM’) WHERE has(persons.name) and (persons.entitytype=’user’) and (persons-[:WORKS_AT]->company) RETURN persons;

// Those persons will be returned who are has relation to ‘IBM’ (Only ‘WORKS_AT’ relationship)

START me=node:node_auto_index(name=’andrew’) MATCH me–>friend where (friend.entitytype=’user’) RETURN count(friend),friend;

// Return only those users who are outbound to ‘andrew’

START me=node:node_auto_index(name=’andrew’) MATCH me<–friend<–friend_of_friend where (friend_of_friend.entitytype=’user’) RETURN count(friend_of_friend),friend_of_friend;

// This is a ‘friend of friend’ relationship. Here ‘John’ is friend of ‘linda’ and ‘linda’ is friend of ‘andrew’

START me=node:node_auto_index(name=’andrew’),company=node:node_auto_index(name=’CTS’) MATCH me<–friend<–friend_of_friend where (friend_of_friend.entitytype=’user’) and (friend_of_friend-[:WORKS_AT]->company) RETURN count(friend_of_friend),friend_of_friend,company;

// This is a ‘friend of friend’ relationship. Here ‘John’ is friend of ‘linda’ and ‘linda’ is friend of ‘andrew’ and he is returned as he is working in ‘CTS’

START me=node:node_auto_index(name=’andrew’),company=node(8,9) MATCH me<–friend<–friend_of_friend where (friend_of_friend.entitytype=’user’) and (friend_of_friend-[:WORKS_AT]->company) RETURN count(friend_of_friend),friend_of_friend,company;

// Same Query of above for selection of all the companies

start n = node:node_auto_index(name=’UI Restructuring’) match n<–x<-[:WORKS_AT]-x1 return x1;

// Here the project is done in ‘IBM’ and nodes – ‘x1’ are working in the company;

start n = node:node_auto_index(name=’UI Restructuring’) match n<–x<-[:FOLLOWS]-x1 return x1;

// Here the project is done in ‘IBM’ and nodes – ‘x1’ are following the company;

START n=node:node_auto_index(name=’jude’) MATCH (n)-[:WORKS_AT]->x<-[:FOLLOWS]-x1 return x1;

// return all the nodes which follows the company on which ‘jude’ is working;

START n=node:node_auto_index(name=’CTS’) MATCH (n)<-[r:WORKS_AT]-() RETURN type(r), count(*);

// How much nodes are in the graph who are with ‘WORKS_AT’ in ‘CTS’

START n=node(*) RETURN distinct(n.entitytype?); // Select distinct entity types

START person=node:node_auto_index(‘name: *’) MATCH person-[?:WORKS_AT]->company where person.entitytype =’user’ RETURN person.name

// return all the nodes with ‘WORKS_AT’ relationship

START begin = node(4), end = node(7) MATCH p = begin -[*]-> end foreach(n in nodes(p) : SET n.network = true);

// Updating connecting nodes and example of loop in cypher query

START begin = node(4), end = node(7) MATCH p = begin -[*]-> end return *; // showing the result of updation in connecting nodes

START n=node(1) MATCH p=n-[*..]->b WHERE ANY(x in nodes(p) WHERE x.network? = true) RETURN p;

// Example of Predicates

References :

1> http://docs.neo4j.org/refcard/1.9/

2> http://java.dzone.com/articles/10-caveats-neo4j-users-should

3> http://docs.neo4j.org/chunked/milestone/cypher-query-lang.html

We have shown some of the queries, which we can use in practical applications in Graph Database. And as this is a new type of NoSql Database, the above post is an introduction for some of the neo4j cypher query examples from our end.

We will try to put the performance and optimisation related issues in Neo4j in our later articles.

So keep reading. Comments are welcome.

Enter your email address:

Delivered by FeedBurner

Expandable List in Android


Today we are going to discuss ExpandableListView in Android.

Here I’ve used Eclipse with ADT plugin to build this example. The example is straightforward.

Create a new Android Application Project in your workspace.

Extend your main Activity from ExpandableListActivity.

For your reference please find the following code :


package com.example.expandablelistviewdemo;
import android.app.ExpandableListActivity;
import android.content.Context;
import android.os.Bundle;
import android.view.LayoutInflater;
import android.view.View;
import android.view.ViewGroup;
import android.widget.BaseExpandableListAdapter;
import android.widget.TextView;
public class MainActivity extends ExpandableListActivity {
private static final String arrGroupElements[] = { "Item1", "Item2", "Item3",
"Item4", "Item5", "Item6", "Item7", "Item8", "Item9" };
/**
* strings for child elements
*/
private static final String arrChildElements[][] = {
{ "Details1 A","Details1 B", "Details1 C" },
{ "Details2 A","Details2 B", "Details2 C" },
{ "Details3 A","Details3 B", "Details3 C" },
{ "Details4 A","Details4 B", "Details4 C" },
{ "Details5 A","Details5 B", "Details5 C" },
{ "Details6 A","Details6 B", "Details6 C" },
{ "Details7" },
{ "Details8" },
{ "Details9" }
};
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.article_screen);
setListAdapter(new ExpandableListAdapter(this));
}
public class ExpandableListAdapter extends BaseExpandableListAdapter {
private Context myContext;
public ExpandableListAdapter(Context context) {
myContext = context;
}
@Override
public Object getChild(int groupPosition, int childPosition) {
return null;
}
@Override
public long getChildId(int groupPosition, int childPosition) {
return 0;
}
@Override
public View getChildView(int groupPosition, int childPosition,
boolean isLastChild, View convertView, ViewGroup parent) {
if (convertView == null) {
LayoutInflater inflater = (LayoutInflater) myContext
.getSystemService(Context.LAYOUT_INFLATER_SERVICE);
convertView = inflater.inflate(
R.layout.article_list_child_item_layout, null);
}
TextView yourSelection = (TextView) convertView
.findViewById(R.id.articleContentTextView);
yourSelection
.setText(arrChildElements[groupPosition][childPosition]);
return convertView;
}
@Override
public int getChildrenCount(int groupPosition) {
return arrChildElements[groupPosition].length;
}
@Override
public Object getGroup(int groupPosition) {
return null;
}
@Override
public int getGroupCount() {
return arrGroupElements.length;
}
@Override
public long getGroupId(int groupPosition) {
return 0;
}
@Override
public View getGroupView(int groupPosition, boolean isExpanded,
View convertView, ViewGroup parent) {
if (convertView == null) {
LayoutInflater inflater = (LayoutInflater) myContext
.getSystemService(Context.LAYOUT_INFLATER_SERVICE);
convertView = inflater.inflate(
R.layout.article_list_item_layout, null);
}
TextView groupName = (TextView) convertView
.findViewById(R.id.articleHeaderTextView);
groupName.setText(arrGroupElements[groupPosition]);
return convertView;
}
@Override
public boolean hasStableIds() {
return false;
}
@Override
public boolean isChildSelectable(int groupPosition, int childPosition) {
return true;
}
}
}

Now we need to create the layout files. Here we’ll create 3 layout files. The first one for the main layout, the second one for the expandable list item and the third one for the expandable list sub item.

Please find the following codes :

Main Layout file (article_screen.xml)


<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:padding="5dp">
<TextView android:id="@+id/articleListHeaderTextView"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_alignParentTop="true"
android:layout_centerHorizontal="true"
android:layout_marginTop="10dp"
android:text="List Header"
android:textAppearance="?android:attr/textAppearanceMedium"
android:textColor="#0585C4"
android:textSize="14dp"
android:textStyle="bold" />
<ExpandableListView
android:id="@android:id/list"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:layout_below="@id/articleListHeaderTextView"
android:layout_margin="10dp"
android:divider="@android:color/darker_gray"
android:childDivider="@android:color/white"
android:dividerHeight="2dp"
android:fadingEdge="none">
</ExpandableListView>
</RelativeLayout>

Expandable list item layout file (article_list_item_layout.xml)


<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:background="#FFFFFFFF"
android:paddingBottom="10dp"
android:paddingLeft="25dp"
android:paddingTop="10dp">
<TextView android:id="@+id/articleHeaderTextView"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_alignParentLeft="true"
android:layout_alignParentTop="false"
android:layout_centerHorizontal="false"
android:layout_centerVertical="true"
android:layout_margin="10dp"
android:textAppearance="?android:attr/textAppearanceMedium"
android:textColor="#0585C4"
android:textSize="12sp"
android:textStyle="bold" />
</RelativeLayout>

Expandable list item child layout file(article_list_child_item_layout.xml)


<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:background="#FFE4DEDE"
android:padding="10dp">
<TextView android:id="@+id/articleContentTextView"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_alignParentLeft="true"
android:layout_alignParentTop="true"
android:layout_centerHorizontal="true"
android:textColor="#FF000000" />
</RelativeLayout>

That’s it. Now you can run this application either on emulator or on real device. It should look like the following images :

Android ExpandableListView example (Expanded)
Android ExpandableListView example (Expanded)
Android ExpandableListView
Android ExpandableListView

Enter your email address:

Delivered by FeedBurner

Case Study in Graph Database in Context of Neo4j Cypher Query



We have presented here a typical case study related to graph database model.

Previously we have written Preliminary concepts in Graph Database.

In this post, we will try to describe a Case Study for a typical Graph Database Model and show the Diagrammatic representation.

Case Study for Graph Database –

Nodes –

Entity – User (Represented as Node)

Entity – Organisation (Represented as Node)

Entity – Project (Represented as Node)

Entity – Skill (Represented as Node)

Relationships –

User worked in Organisation. – (Relationship – worked)

User is following Oganisation – (Relationship – following)

User has done Projects – (Relationship – has done)

User has Skills – (Relationship – has)

User is Friend of User – (Relation – is Friend)

Diagrammatic Model for Graph Database Representation –

Graph Database Model

In next Post we described Neo4j Cypher Query to represent the above model.

So keep visiting. Comments are welcome.

Enter your email address:

Delivered by FeedBurner

Preliminary Concepts of Graph Database


We will try to clarify the Concepts of  Graph Database in context of neo4j – a graph database in our posts one-by-one.

So below is our first post which describes the quick start up concepts of Graph Database.

Graph –

A graph consists of information in Nodes which are interconnected. A Graph can consists of a single node also.
A particular node can have multiple properties in it as key-value pair. Also the nodes are connected in multiple ways,
which we can describe as relationship.

Examples –

A School information is a single node.
Students and Teachers are different nodes which are connected through relationship.

Students(Nodes) study(Relationship) in School(Node).

Teachers(Nodes) teach(Relationship) in School(Node).

School has name, address, year of stablishment, accreditation etc. which can be described as Properties.

Student has name, address, year of join, class etc. which can be described as Properties.

Teacher has name, address, year of join, highest qualification etc. which can be described as Properties.

Node

Nodes are every single set of entity information in a Graph.

Examples –

If we consider a school information system as a Graph,

School itself is a node in the School Graph.

Each Student is node in the School Graph.

Also, each Teacher is node in the School Graph.

Relationship

Nodes are joined by some information – which are relationship in graph.

Examples –

Students study in School.

Teachers teach in School.

Here study and teach are the relationship information between nodes in a School Graph.

Labels – When nodes form a group in a graph and are identified by some information, then that information will be called label.

Examples –

In a school graph,

we can classify

(All Students) label for all the students in the school graph.

Again,

(5th Class Students), (6th Class Students) label can be given for respective students.

Here (All Students), (5th Class Students), (6th Class Students) can be called as sub graphs.

Node Traversal –

When we look for some specific information in a graph of all the Nodes, we process the information udentsin some specific pre-determined optimum way
and this is called Graph Traversal.

Examples –

In a school graph, to have a information like “Name of the teachers who teaches the 5th Class students”,

We can traverse by

In (Paricular SChool) –>> study by –>> (5th Class Students) –>> Teached by –>> (Teachers)

Note : Above is a traversal information, which ma y be used for understanding traversal in a Graph.

Indexes –

When we want for a specific set of information of nodes or relationship, we put all those information in specific maps, which will be called as index.

Examples –

In a school graph-

We can have specific set of nodes for

(Science Subject Teachers), (Art Subject Teachers) etc. which we can put in indexes.

In next part of our posts related to Graph Database,

we will describe  a Case Study where we can use a Graph Database.

Then,

We will describe the schema and information we can obtain from Graph Database through Cypher Query in context of neo4j.

The diagrammatic representation of the Graph Database is here.

Most of the queries with Neo4j Cypher Language for the Diagram are explained in this post.

So keep visiting here. Comments are welcome.

Reference –

What is a Graph Database ?

Enter your email address:

Delivered by FeedBurner

Keyword Selection – backbone of SEO


SEO is the abbreviation of Search Engine Optimization.Overall SEO process is described in Search Engine Optimisation – A Small Brief.

Here we will discuss about the relevancy of KEYWORD in SEO. Keyword is the one of the important property of SEO. The search string/term entered by the net surfers, is called KEYWORD.

Most of the time the keyword consists of two to five words.

Choosing the right keyword for search in relevant content is the first & most important step to a successful SEO campaign. A lot of research work need to be done on keyword to get the right keyword and a list of keywords is constructed, after careful analysis of what the online population are searching for.

 

Choosing the right keyword:

Near about ten years back, the keyword might consists of one word. But now a days, as the net is so densely populated with websites, it is impossible to gain good rank by that. So, keyword should contain more than one word or a phrase to get a good rank in SEO.

Like, if someone use the keyword “computer”, a lots of result will be similar to that as there is a large number of pages exist with this keyword. So automatically the rank will be much lower. Now if, some phrases are added with this, like, “fastest computer” or “computer with higher memory”, the search result list will automatically narrowed down. So, for keyword one should use a word or phrase which is more specific & highly popular among the regular net surfers. The keyword also needs to be closely related with the content of the page.

 

Keyword Frequency:

There are some rules regarding the occurrence of keyword. The keyword should exist in the title or content of a web page. If the frequency is much less, then it will be hard to find that page by search engine with the searched string. It is calculated that 3-7 time occurrence of a keyword is well enough. In contrary if frequency is much higher, i.e. if the content of a page contains a specific word too many times, search engine will assign a low rank. This is called page stuffing or spamming.

Some search engine also consider the keyword weight. The total number of keyword present in a web page, compared to the total word count is called keyword weight. So the total number of words & total number of keywords should have a clean ratio.

 

Where we will insert keywords in a content?

Till now we have learned how keyword plays an important roll in SEO. To gain a good rank for a website, it is also important to know exactly where to put keywords in a webpage.

Below some places have been categorized where keywords can be placed.

a) Keywords in URL: Keywords can be put in the url of a page. Like, if a page contains information about the Advance JAVA, the url can be http://ourjavasite.com/advance_java.  Though from SEO perspective url should contain 5 keywords in the url, still normally 3 keywords in url is sufficient.

b) Keyword in file name: Search engines also crawl a page with its file name. So keywords can be inserted in the file name. So if the url of JSP content is http://ourjavasite.com/jsp rather than http://ourjavasite.com/java, it is always better.

c) Keyword in page title: Page title is another place which search engines displays as search result. We can insert keyword in the page title tag i.e. (<title></title>).

d) Keyword in headline: Headline tags (<h1>,<h2>) are also a special portion of a page that search engine use to crawl.

It is to be noted that all these depends on the search engine.

For some search engine, keyword in title achieve more preference than keyword in heading & vice versa for some others.

Enter your email address:

Delivered by FeedBurner

Search Engine Optimisation – A Small Brief


SEO Components
SEO Components

 

What is SEO?

SEO stands for Search Engine Optimization. A search engine is a very common thing every net surfer faces now a days. By using search engine one can find some article or topic by using a word or more than one word in a Search Engine Interface. In SEO, the searching terms are called Keywords.

SEO is the activity of optimizing Web pages of a whole site in order to make them more search engine friendly, thus getting higher positions in search results in search results.

Examples for search engine are GOOGLE, BING, YAHOO etc.

The objectives of SEO are,

a) Optimizing a web site for search engines.

b) Processing the designing & developing job of a website to get a well rank in search engine results.

c) Improve the volume & quality of traffic of a website from search engines.

d) It is a fact for understanding how search algorithms work & what human visitors want to see.

 

What is the working mechanism of SEO?

Now we should understand to know – how SEO works.

The total working mechanism of SEO can be divided into 5 steps.

a) Crawling: Crawling is the process of fetching all the pages of a website.  All the linked pages of a web site is fetched using a software called crawler or spider. Like, Googlebot for google.

b) Indexing: Indexing is the process where all the pages of websites are organised & they are kept in a giant database where it can later be retrieved. The word or keyword entered by the user will be used to match the best described pages using the indexing.

c) Processing: The search string(keyword) is processed in this step. It means, the search string is compared with the indexed pages.

d) Calculating the relevancy: Since it is likely the search string can be matched with more than one page at a time, search engine starts to calculate the relevancy of each page from the indexing to the search string.

Top search engines like, GOOGLE, BING revise their relevancy algorithms often. The change in ranking of a website in search results, is due to the change in relevancy algorithm.

e) Retrieving the results: In the last step of search engine, the best matched pages are fetched from the database, and showed them in the browser.

Types of SEO tactics – White Hat SEO & Black Hat SEO

White Hat SEO:

Technique that search engine recommends as part of good design and content is referred as White Hat SEO technique.

Following properties decide SEO as White Hat SEO:

a) If a web page contains good design.

b) If a search result satisfies all search engine guidelines.

c) If it ensures the content of a page, that search engine indexes, is the same – which users will see.

d) If the content of page is of good quality and useful information.

Black Hat SEO:

Techniques that search engine do not approve or try to minimize the effect of page in the search results, is called Black Hat SEO.

A page can be enlisted in Black Hat due to the following reasons,

a) If a web page does not satisfies the search engine guidelines.

b) If the contents of a web page contains a small or inferior amount of information.

c) If the page is stuffed with a certain word or phrase which is called Keyword Stuffing.

d) If a web page page looks like a well ranked web page, but it redirects to some unrelated pages which does not contain useful information.

e) If the meta tag contains repeated keywords with are unrelated to the search keyword.

f) If a page produce one version to the search engine & another version to the human visitors.

Search engines are smart enough to track all the above properties of a web page.

 

Human psychology in SEO

Search engines are smart enough to follow human psychology during indexing the search results & displaying it in the browser.

There are some interesting facts about human psychology during surfing of the search results from the list. Some of those are like –

  1. When a keyword is entered as search string, search engine process it, and show the relevant search results. From the list, users will pay more attention to the 1st searched result in the list by default. This happens for almost every user & some time will be spent for the 1st link.
  2. If the user do not find the 1st result much useful, attention will converted to the 2nd result of the list. They will again spent some time there.
  3. Now a very interesting thing happens. If the user tends to surf some more, typically the 3rd result from the list will be skipped & 4th result will be opened.
  4. When the browser shows some results according to the searched string, its human nature to see the top left corner of the browser, then top right & then left side, unintentionally ignoring the right side of the browser.

SEO shows the searched results by considering these psychological facts. The most popular or top rated link according to the searched string is placed in the 2nd position of the list. According to the ranking the results are shown in the list. The 3rd position is replaced will comparatively low ranked link.

The position of the result list is placed on the left side vertically. And on the right side some less relevant information or advertisement is placed.

 

Designing in SEO:

A web site with a colorful design automatically attract users. A web site with user friendly interface & easy to understand operations always welcomed by the regular net surfers.

A search engine is a software not any human being, that it will read the contents with its own interest. So, if the website is too complicated to operate, search engine fails to parse the content. Indexing will be poor, as a result it will be assigned to a low rank.

Designing of website should follow some guidelines:

  1. The web pages must contain more text than html tags.
  2. Use of frames should be avoided.
  3. Javascript creates problems as the drop-downs created with Javascript can not be parsed by crawler.
  4. Number of advertisement should be less.
  5. Anything irrelevant with the page topic should be removed. There should be no doubt in the search engine’s mind about the content of the page.
  6. No unnecessary stuffs shoul be there (like flash, animated images etc).

 

Keywords

Keywords are the most & ultimate important thing for SEO. The keywords are used to search contents from the huge database of webpages. Web pages are indexed based on the searched topics or keyword & displayed on the browser. Ranking of a web site is dependent on the proper usage of keywords.

The relevance of  Keyword in SEO will be discussed in the next article.

Also what we have missed ? Keep commenting here.

Enter your email address:

Delivered by FeedBurner

Developing Web Application with Node.js, Express.js and MongoDB (MongoJS module)


This is our second proof of concept towards make a web application with Javascript based Web Server.

To make this possible, we have selected –

1> Node.js for server side development

2> Rest based web service creation with express.js

3> Database – MongoDb

4> Node.js MongoDb Module Extention (mongojs)

We have created a Proof of Concept with Javascript based web server, where we have focused on dealing with NoSql (MongoDB) with javascript based framework Node.js.

Our initial code can be downloaded here.

Architecture at a glance –

Node-Express-Mongo

So here are the steps –

Installation –

A> Download and install Node.js from here.

B> To Develop the application we need to install mongojs module for Node.js

Command – npm install mongojs (should be connected to internet)

C> We need to install express.js for node.js

Command – npm install express  (should be connected to internet)

Configuration Code –

Now, we will try to describe the code portion –

 
var application_root = __dirname,
    express = require("express"),
    path = require("path");

Here we have initialised the express.js within javascript variables in respect of Node.js concept.

 
var app = express();

Here we have initialised the express web server in app variable.

 

var databaseUrl = "sampledb"; 
var collections = ["things"]
var db = require("mongojs").connect(databaseUrl, collections);

Here we have made the connection to the mongodb database using the Node.js mongojs module extension library.

 
// Config

app.configure(function () {
  app.use(express.bodyParser());
  app.use(express.methodOverride());
  app.use(app.router);
  app.use(express.static(path.join(application_root, "public")));
  app.use(express.errorHandler({ dumpExceptions: true, showStack: true }));
});

Here we have made the configuration related to express.js

Rest Services Code –

 
app.get('/api', function (req, res) {
  res.send('Our Sample API is up...');
});

Here we have made our first REST based web service and tested whether the express.js is up.

Our sample api will be  – http://127.0.0.1:1212/api (Get Method)

 
app.get('/getallusers', function (req, res) {
	db.things.find('', function(err, users) {
	if( err || !users) console.log("No users found");
	  else 
	{
		res.writeHead(200, {'Content-Type': 'text/plain'});
		str='';
		users.forEach( function(user) {
			str = str + user.username +'\n';
		});
		res.end( str);
	}
  });
});

Here we have created another REST api to get all username from user collection and so have done the mongojs query.

Our sample api will be  – http://127.0.0.1:1212/getallusers (Get Method)

 
app.get('/user/:username', function (req, res){
	res.writeHead(200, {'Content-Type': 'text/plain'});
	//res.end( req.params.username);
	user = req.params.username;
	db.things.find({username:user}, function(err, users) {
		 str='';
		 if( err || !users) console.log("No users found");
		 else 
		{
			users.forEach( function(user) {
				str = str + 'User is '+ user.username +'\n';
				str = str + 'and email is '+ user.email +'\n';
				res.end( str);
			});
		 }
      }); 
});

Here we have selected particular user by his/her username in database via REST calling.

Our sample api will be  – http://127.0.0.1:1212/user/firstuser (Get Method)

 
app.post('/insertmongouser', function (req, res){
  console.log("POST: ");
  res.writeHead(200, {'Content-Type': 'text/plain'});
  user = req.body.username;
  passwd = req.body.password;
  emailid = req.body.email;

  db.things.save({email: emailid, password: passwd, username: user}, function(err, saved) {
  if( err || !saved ) res.end( "User not saved"); 
  else res.end( "User saved");
});
});

Here we have made a POST request to create an user via REST calling.

Our sample api will be  – http://127.0.0.1:1212/insertmongouser (Post Method)

 
// Launch server
app.listen(1212);

We have made the server to listen at 1212 port. Also below is the samplemongodb.html file which is used to call the REST Post API to insert the user.

 

<html>
<body>
<form name="input" action="http://127.0.0.1:1212/insertmongouser" method="post">
Username: <input type="text" name="username">
Password: <input type="text" name="password">
Email: <input type="text" name="email">
<input type="submit" value="Submit">
</form>
</body>
</html>

Now run node appmongodb.js from command shell.

Enter your email address:

Delivered by FeedBurner

A Quick Primer on MongoDB


We have used some of useful commands in MongoDB in our project work. Below is the quick reference of MongoDB commands which can be executed from MongoDB shell. We have tested all the commands from mongodb shell with test data examples.

We can download mongodb from here.

To start mongodb

we should use –

mongod –dbpath <<Our MongoDB Data Directory>>

Now to start the Database from shell

mongo // to satrt the db

db // to test current database

show dbs // to show all the databases

use userdb // to change to database or to create a new database

// Now to insert data in Collection

user = { username : “firstuser”,password:”firstuser”,email:”firstuser@gmail.com” } // prepare the data

db.users.insert(user); // insert the data in things collection

show collections; // to see all the collections in mongodb, here we should view the collection “users”  which we have previously created

db.users.find(); // to see all the documents in the collection

db.users.find(0); // find collection by index

db.users.find({username:”firstuser”}); // find doucment in collection by json elements

//Working with cursors in json –

var c = db.users.find();
while ( c.hasNext() ) printjson( c.next() )

// CRUD Operation –

// Insertion of Data

user = { username : “firstuser”,password:”firstuser”,email:”firstuser@gmail.com” } // prepare the data

//Below command can be used to insert with “_id” or without “_id”

db.users.insert(user); // insert the data in things collection.

//To insert data without “_id” field

db.users.save({username:”thirduser”,password:”thirduser”,email:”thirduser@gmail.com”});
db.users.find(); // Confirming the operation

// To insert data in collection with “_id” field if the “_id” not found

db.users.save({_id:10,username:”fourthuser”,password:”fourthuser”,email:”fourthuser1@gmail.com”});

// To update data in collection with “_id” field if the “_id” found

db.users.save({_id:10,username:”fourthuser1″,password:”fourthuser1″,email:”fourthuser1@gmail.com”});

// Same operation update with upsert flag i.e. if data found with “_id” then update operation else insert operation
// follow the opetation – if username with “fifthuser” is found then update operation else insert operation

db.users.update({username:”fifthuser”},{_id:11,username:”fifthuser”,password:”fifthuser”,email:”fifthhuser@gmail.com”},{upsert:true});
db.users.find(); // data insertion should be there

db.users.update({username:”fifthuser”},{_id:11,username:”fifthuser1″,password:”fifthuser1″,email:”fifthhuser1@gmail.com”},{upsert:true});
db.users.find(); // data updation should be there

// Select data from mongodb

db.users.find({username: { $in: [ 10, 11 ] }});
db.users.find({username: { $in: [ “firstuser”, “seconduser” ] }});

// A query cursor with find operation –

var myCursor = db.users.find( { username: /.*user.*/ } );
myCursor.forEach(printjson);
// Update field in document
// Set a new field

db.users.update({ _id: 10 },{$set: { ‘username’: ‘sixthuser’ },});
db.users.update({ _id: 10 },{$set: { ‘useraddress’: ‘sixthuser address’ },});
db.users.find();

// Unset a field in the document

db.users.update({ _id: 10 },{ $unset: { useraddress: ‘sixthuser address’ } });
db.users.find();

// Remove document from collection

db.users.remove( { ‘username’ : ‘fifttuser1’ } );

Reference links –
http://docs.mongodb.org/manual/core/update/
http://docs.mongodb.org/manual/core/delete/
http://docs.mongodb.org/manual/core/read/
http://docs.mongodb.org/manual/core/create/

Enter your email address:

Delivered by FeedBurner

Installation and running Apache Nutch and Apache Solr for crawling and indexing Web Content


In our work, we needed to use open source web crawler for unstructured data gathering.

Here we have used

A> Apache Nutch for web crawling and

B> Apache Solr for unstructured web data indexing

Steps, that we have used to set up the complete environment are –

1> Downloaded Apache Solr (3.X)

2> Downloaded Apache Nutch ( 1.x)

We followed the tutorial from here.

We are giving here all the steps with the most important points, for which we have spent much time. Our OS was Mac OS X.

In Linux Terminal, Setting of two Environmental variables are important.

1> JAVA_HOME

export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/

2> NUTCH_JAVA_HOME

export NUTCH_JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/

Checked that the variables are set properly.

Apache Nuch Installation

Unzip Apache Nutch.

Run ‘bin/nutch’ to check whether nutch is installed properly or not.

Goto <<apache nutch installation directory>>/conf folder.

Open nutch-site.xml file.

save with following XML snippet:

<configuration>

<property>

 <name>http.agent.name</name>

 <value>My Nutch Spider</value>

</property>

</configuration>
  • Create folder urls in <<apache nutch installation directory>>
  • Create a file seed.txt
  • Enter your required site to crawl. In our case it was http://blog.phloxblog.in
  • Execute following command in terminal –

      bin/nutch crawl urls -depth 5 -topN 20

     Here,

     depth – depth indicates the link depth from the main url that should be crawled.

     topN – N determines the maximum number of pages that will be retrieved at each level up to the depth.

Now,

during crawling, it will create  folder named like crawl-(timestamp)

Within crawl-(timestamp) folder, there will be  3 folders named – crawldb, linkdb and segments.

In crawldb folder, there will be ‘current’ folder. In the current folder, there will be ‘part-00000’ folder and within that another two file – data & index. 

So all the crawled data will be stored in  crawl-(timestamp) folder, if everything goes fine.

troubleshooting – If you get unknownhostexception in data file,

then

1> check the internet connection.

2> follow the versions and steps again as we have mentioned.

Note that, we have changed the command –

bin/nutch crawl urls -depth 5 -topN 20 which is slightly different from nutch wiki documentation.

So change the command as per your requirement, after reading the nutch command usage here.

Apache Solr Installation –

  • Unzip Apache Solr
  • Run  java -jar start.jar from <<Apache Solr Directory>>/example directory.
  • It will start solr in the 8983 port for default Apache Solr Installation.
  • Hit the url http://localhost:8983/solr/admin/. to check wheher solr installation is fine.
  • Close Apache Solr.

Note : for detail Apache Solr Installation you can refer here.

Integration of Apache Nutch with Apache Solr –

To integrate solr with nutch. 

  1. Copy schema.xml from conf folder of <<apache-nutch installation directory>>.
  2. Go to the <<apache solr installation directory>> . On that folder open the path example>solr>conf.
  3. Replace schema.xml with the copied one. 
  4. Open the schema.xml file. Change the line,
     <field name="content" type="text" stored="true" indexed="true"/> to store the crawled content as indexed.

Execute the following command from terminal in <<Apache Nutch Installation Directory>> 

bin/nutch solrindex http://127.0.0.1:8983/solr/ <<the folder name that has been created during crawling-crawl-timestamp>>/crawldb -linkdb crawl/linkdb <<the folder name that has been created during crawling-crawl-timestamp>>/segments/*

This will complete indexing of all the crawled data from nutch.

To be sure if the above process is done perfectly or not,

It will show all the data that have been indexed after crawling.

So happy crawling. Comments are welcome.

Enter your email address:

Delivered by FeedBurner