Git Repo to Neo4J

Here is a little project (~ 2 days of development), playing around with both JGit and Neo4J (and springboot-data-neo4j).

It read Git Commits/RevTree/Person/… graph from a local Git repository, and update the corresponding Graph objects in Neo4J.
So you can display it graphically, and perform Cypher requests on it.


I have tested it on the JGit repository itself … </img>

The source code repository “gitrepo-to-neo4j” is open-source, and you can found it here: Github test-snippets/gitrepo-to-neo4j

You could try displaying All objects…but it is too much for one screen.

Java Libraries

JGit

JGit = a java library for accessing a Git repository

Neo4J

Neo4J = a graph database (with java driver client, rich Web graphical client (d3s..), a Rest api, ..)
The fact is that Neo4J server is also written in java make it a breeze to download+unzip+start.

Springboot-data-neo4j

springboot-data-neo4j = springboot magic for Neo4j …

Motivations...

It is fun to code, at least more fun that playing stupid video games, like many students I know…

More seriously, at work there are complex reporting tools trying to check informations between SVN/GIT and Jira, and then between Jira and a deployment tool (XLDeploy from XebiaLabs)…

some One of the goals is to check that a developped feature as specified in a Jira Issue, is deployed on Developments/Tests/Prod environments.

Internally, a top level Jira feature request has many corresponding Jira sub-issues per deployable components, which corresponds to many GIT commits (the Jira issue id is mandatory in the git commits message ..). Then jenkins builds are linked to Git commits, and produce deployable versions… which are deployed using the deployment tool…

These reporting tool is quite a mess.

It was fun re-looking at it as a unification Graph database of both GIT commits graph and Jira Issues graph. This was the subject of a team lunch-time Coding DOJO, as I organised them every week at work.

Getting started with Neo4J

  • Download
  • Start just type "./bin/neo4j start" ... or if you are running on windows, and not a Windows local administrator, (cannot start a windows Service) => just use ".\bin\neo4j.bat console"
  • open your browser on http://localhost:7474/browser All you have to do is to change your password on first login
  • there is also a nice running demo embedded in the UI

Some links:

Some Example of Cypher commands:

match(r:SymbolicRepoRef) return r

match(c: RevCi) return c limit 100

match(u:Person) return u limit 100

match(c: RevCi)-[r]-(t) return c,r,t limit 100

match(c: RevCi)-[p:parent]-(c2: RevCi)  return c,p,c2 limit 100

Here is an extract example of Java code for inserting / reading Person in Neo4J graph database

@NodeEntity(label="Person")
public class PersonIdentEntity {

	private Long id;
	
	private String name;
	private String emailAddress;
	
	// getter-setter ommited .. use Lombok some day
}

import org.springframework.data.neo4j.repository.GraphRepository;
import fr.an.tools.git2neo4j.domain.PersonIdentEntity;

/** DAO=Repository ... magically created/injected by springboot-data */ 
public interface PersonDAO extends GraphRepository<PersonIdentEntity> {
}

@Component
public class Git2Neo4JSyncService {

	@Autowired
	private PersonDAO personDAO;

	@Transactional
	public void syncRepo(Git git) {
	    // find all persons from Neo4j graph db
		Iterable<PersonIdentEntity> personEntities = personDAO.findAll();
		
		// traverse jgit RevCommit graphs
		org.eclipse.jgit.lib.RevCommit revCommit = ..   
		org.eclipse.jgit.lib.PersonIdent commitAuthor = revCommit.getAuhor();
		
		// if already created, find personEntities by email
		PersonIdentEntity personEntity = ..  
		if (personEntity == null) {
		  // create corresponding Neo4j person entity
		  personEntity = createPersonEntity(commitAuthor);
		}
	}	
	
	protected PersonIdentEntity createPersonEntity(PersonIdent src) {
		PersonIdentEntity res = new PersonIdentEntity();
		res.setEmailAddress(email);
		res.setName(src.getName());

		return personDAO.save(res);
	}
}

Getting started with JGit

Of course, it is mandatory to understand GIT Internals, before trying to use JGit. I recommend