Jane's Technical Stuff
Archives
May 2002March 2003
May 2003
July 2003
August 2003
June 2005
July 2005
August 2005
December 2005
January 2006
May 2006
June 2006
July 2006
August 2006
September 2006
October 2006
November 2006
December 2006
January 2007
February 2007
March 2007
April 2007
May 2007
June 2007
July 2007
August 2007
September 2007
October 2007
November 2007
December 2007
January 2008
February 2008
March 2008
April 2008
May 2008
June 2008
July 2008
August 2008
September 2008
October 2008
Wednesday, September 03, 2008
Database Comparison tools: Redgate SQL Compare Review
I first started using Redgate SQL Compare when I was at Glass's in 2003/04 and I'm still using it 4 years later. Over those years the product has evolved and new products have been released from Redgate. I have only ever used it for SQL Server (7, 2000 and 2005) and so can't comment on it's appropriateness or reliableness targeting other database products.
At the time that I first found SQL Compare, there weren't that many tools available that did a reliable job of comparing objects and producing scripts of the differences. I know that one of my colleagues at Madgex is a fan of SQL Delta. One day we sat down to compare the tools and discovered that there really wasn't much to choose between them.
I have always used SQL Compare more as a tool to check my scripted updates, rather than to produce those scripts. After a bad experience at a previous company when a member of my team managed to empty an articles table rather than update it when manipulating the database via Enterprise Manager I've habitually manually created defensive SQL change scripts with appropriate transactions and error handling.
SQL Compare is simple to use, works quickly and produces a detailed list of differences between database objects. The differences that are detected are configurable, so you can choose to ignore or report upon white space or comment differences.
When you first open the screen you are presented with a screen offering options of what you want to compare.
Once you have registered a couple of databases to compare, it is a short wait until the differences are displayed.
The display of differences are divided into:
For each object that is different you can choose to view the details of that object which will display a SQL based description of the object with the differences identified by a highlight line - sometimes it might be the whole object, a part of an object (i.e.a column definition) and sometimes it will be a constraint, or a grant statement that is missing.
From the summary screen you can select the changes that you're interested in and get SQL Compare to automatically synchonise your target database to match the source, or vice versa. You can also get it to generate some SQL to do the generation to allow you to put your SQL upgrade scripts into a build process, or under source control. In the majority of cases I will use this tool to check my upgrade scripts, rather than to produce them and if I spot any difference I tend to write my own, defensive, SQL to make the upgrade scripts.
In the 4 or so years I've been using SQL Compare I've found it to be a stable, evolving product which I have come to rely upon completely to check upgrade processes. I made the most use of this tool at Glass's where I was working on a product which was client/server application where it couldn't be relied upon that the customer was always running the latest version. Consequently, our preparation for release process involved installing the earliest supported version of the system, and upgrading it to the release candidate on one machine and installing a clean, new build of the release candidate from scratch and comparing the two and producing upgrade SQL scripts for any identified differences before starting the process again. We also included a call to the command line version into the automated MSBuild process producing an HTML report of the differences which could be used to fail the build if anything other than 0 differences was reported.
At the time that I first found SQL Compare, there weren't that many tools available that did a reliable job of comparing objects and producing scripts of the differences. I know that one of my colleagues at Madgex is a fan of SQL Delta. One day we sat down to compare the tools and discovered that there really wasn't much to choose between them.
I have always used SQL Compare more as a tool to check my scripted updates, rather than to produce those scripts. After a bad experience at a previous company when a member of my team managed to empty an articles table rather than update it when manipulating the database via Enterprise Manager I've habitually manually created defensive SQL change scripts with appropriate transactions and error handling.
SQL Compare is simple to use, works quickly and produces a detailed list of differences between database objects. The differences that are detected are configurable, so you can choose to ignore or report upon white space or comment differences.
When you first open the screen you are presented with a screen offering options of what you want to compare.
Once you have registered a couple of databases to compare, it is a short wait until the differences are displayed.
The display of differences are divided into:
- objects that appear on the source database but not the target
- objects that appear on the target database but not the source
- objects that appear on both databases but are different
- objects that are the same on both databases
For each object that is different you can choose to view the details of that object which will display a SQL based description of the object with the differences identified by a highlight line - sometimes it might be the whole object, a part of an object (i.e.a column definition) and sometimes it will be a constraint, or a grant statement that is missing.
From the summary screen you can select the changes that you're interested in and get SQL Compare to automatically synchonise your target database to match the source, or vice versa. You can also get it to generate some SQL to do the generation to allow you to put your SQL upgrade scripts into a build process, or under source control. In the majority of cases I will use this tool to check my upgrade scripts, rather than to produce them and if I spot any difference I tend to write my own, defensive, SQL to make the upgrade scripts.
In the 4 or so years I've been using SQL Compare I've found it to be a stable, evolving product which I have come to rely upon completely to check upgrade processes. I made the most use of this tool at Glass's where I was working on a product which was client/server application where it couldn't be relied upon that the customer was always running the latest version. Consequently, our preparation for release process involved installing the earliest supported version of the system, and upgrading it to the release candidate on one machine and installing a clean, new build of the release candidate from scratch and comparing the two and producing upgrade SQL scripts for any identified differences before starting the process again. We also included a call to the command line version into the automated MSBuild process producing an HTML report of the differences which could be used to fail the build if anything other than 0 differences was reported.
Labels: database change management, review, SQL, SQLServer2005, tool
// posted by Jane @ 9:36 AM 
Comments:
Wednesday, August 27, 2008
Review: Clone Detective
The Morning Brew #162 mentioned a tool Clone Detective which
Today, a colleague and I spent some ILP time investigating the tool to determine whether it would help us in the Madgex environment or not.
We tried the tool out on a couple of projects, but due to it being a Visual Studio 2008 only plug-in our options were a bit limited. After installing the plug-in, the next thing we did was watch the video. This gave us a good overview of how it works, and where to find it (hiding under View -> Other Windows -> Clone Explorer). During the video it explained that effectively this tool breaks code down into a series of tokens, and then looks for other pieces of code which can also be broken down into the same series of tokens. The terminology took a little while to get a grasp of - there are clones and there are clone classes. A clone class is a series of tokens which is/may be repeated throughout the solution. A clone is an instance of this fragment. So, if there was a line of code which did something like a = b + c, the clone class would be [var] = [var] + [var], and the clones could be lines of code which are a = b + c or d = e + f (irrespective of whitespace or variable names etc).
It works across an entire solution, so we set it running and looked at the clones it found. Unfortunately, a lot of the clones that were detected for us were false positives - like properties which when tokenised are the same, but which can't really be re-factored. Additionally, some of the clone classes that it detected, are actually multiple presentations of the same code. For instance, one file I looked at had the following lines all marked as clones of different clone classes:
The interface is easy to get to grips with, and is accessed either via its own panels (Clone Explorer, Clone Intersections and Clone Results) or by an indication in the code window with a context sensitive menu item Find Clones, or Show Clone Intersections. Clicking on an item in the explorer allows you to navigate into more detail and find the clone class or find all instances of it. In a big solution, with a lot of clones (false positives or otherwise) the right click to get clone class listing can take quite a while - in the order of 10 - 20 seconds - which can result in lots of re-right-clicking when one is impatient. This may indicate a problem with scalability.
The tool would have proved a lot more useful to us with a few extras:
So, our summary is, a nice idea but gives too many false positives and isn't refined enough at the moment to be of use within our environment on a regular basis. It may, however, still be useful as a one-off exercise (with patience) to find the key areas that should be addressed. In a less complex code base it might be worth another look. Additionally, if when working on some code we detect something that looks like its been copied and pasted, then it might well be worth running Clone Explorer to find other areas that could benefit from our refactoring.
This leaves us with the question, how else can we detect clones in our code base? Until we find something else, this is better than nothing. Does anyone have any suggestions of other tools which could address this area?
is a Visual Studio integration that allows you to analyze C# projects for source code that is duplicated somewhere else. Having duplicates can easily lead to inconsistencies and often is an indicator for poorly factored code.
Today, a colleague and I spent some ILP time investigating the tool to determine whether it would help us in the Madgex environment or not.
We tried the tool out on a couple of projects, but due to it being a Visual Studio 2008 only plug-in our options were a bit limited. After installing the plug-in, the next thing we did was watch the video. This gave us a good overview of how it works, and where to find it (hiding under View -> Other Windows -> Clone Explorer). During the video it explained that effectively this tool breaks code down into a series of tokens, and then looks for other pieces of code which can also be broken down into the same series of tokens. The terminology took a little while to get a grasp of - there are clones and there are clone classes. A clone class is a series of tokens which is/may be repeated throughout the solution. A clone is an instance of this fragment. So, if there was a line of code which did something like a = b + c, the clone class would be [var] = [var] + [var], and the clones could be lines of code which are a = b + c or d = e + f (irrespective of whitespace or variable names etc).
It works across an entire solution, so we set it running and looked at the clones it found. Unfortunately, a lot of the clones that were detected for us were false positives - like properties which when tokenised are the same, but which can't really be re-factored. Additionally, some of the clone classes that it detected, are actually multiple presentations of the same code. For instance, one file I looked at had the following lines all marked as clones of different clone classes:
- lines 20 - 43
- lines 20 - 59
- lines 24 - 43
- lines 24 - 44
- lines 37 - 59
The interface is easy to get to grips with, and is accessed either via its own panels (Clone Explorer, Clone Intersections and Clone Results) or by an indication in the code window with a context sensitive menu item Find Clones, or Show Clone Intersections. Clicking on an item in the explorer allows you to navigate into more detail and find the clone class or find all instances of it. In a big solution, with a lot of clones (false positives or otherwise) the right click to get clone class listing can take quite a while - in the order of 10 - 20 seconds - which can result in lots of re-right-clicking when one is impatient. This may indicate a problem with scalability.
The tool would have proved a lot more useful to us with a few extras:
- Ability to ignore properties
- Ability to list all clone classes at once, rather than having to right click on a file in the Clone Explorer and choose Show Clone Intersections. Then picking a file in there and right clicking to select Clone Class x -> Find All Occurrences
- Ability to mark clone classes as "ignore" so it doesn't report on them again
So, our summary is, a nice idea but gives too many false positives and isn't refined enough at the moment to be of use within our environment on a regular basis. It may, however, still be useful as a one-off exercise (with patience) to find the key areas that should be addressed. In a less complex code base it might be worth another look. Additionally, if when working on some code we detect something that looks like its been copied and pasted, then it might well be worth running Clone Explorer to find other areas that could benefit from our refactoring.
This leaves us with the question, how else can we detect clones in our code base? Until we find something else, this is better than nothing. Does anyone have any suggestions of other tools which could address this area?
// posted by Jane @ 5:26 PM
Comments:
Hi Jane,
I am one of the developers of the clone detective detection algorithm, which is part of the ConQAT (http://conqat.in.tum.de) analysis toolkit. (The plugin you reviewed is an integration into Visual Studio to make it easy to use in the .NET community, but the actual detection is performed by ConQAT).
Thanks for the review! I would like to comment on some of the feature requests you mentioned, since some of them are already supported by ConQAT, even though they are not already included in the VS integration.
1.) Tailoring: detection can be tuned, i.e. to ignore properties or exclude overlapping clone classes as you mentioned in the post.
2.) Overview: ConQAT's primary use case is continuous analysis. To be able to be run non-interactively, it writes its results to HTML pages. These pages also include lists of all clones, treemap-visualizations that display the distribution of the clones over the system or trend charts to evaluate the development of clones over time.
3.)Blacklisting: clones can be blacklisted, so that they do not show up in future detection runs.
Furthermore, ConQAT supports clone detection for additional languages, more specifically C/C++, VB.NET, Java, Cobol and PL/1. (And a bunch of other analyses beyond clone detection, but this is probably out of scope here)
You can find documentation of ConQAT here http://conqat.in.tum.de and on clone detection specifically here: http://conqat.in.tum.de/index.php/CloneDetectionTutorial. We are still lagging behind on documentation somewhat though, so feel free to contact me if you have any questions.
I hope that helps, Best regards,
Elmar
I am one of the developers of the clone detective detection algorithm, which is part of the ConQAT (http://conqat.in.tum.de) analysis toolkit. (The plugin you reviewed is an integration into Visual Studio to make it easy to use in the .NET community, but the actual detection is performed by ConQAT).
Thanks for the review! I would like to comment on some of the feature requests you mentioned, since some of them are already supported by ConQAT, even though they are not already included in the VS integration.
1.) Tailoring: detection can be tuned, i.e. to ignore properties or exclude overlapping clone classes as you mentioned in the post.
2.) Overview: ConQAT's primary use case is continuous analysis. To be able to be run non-interactively, it writes its results to HTML pages. These pages also include lists of all clones, treemap-visualizations that display the distribution of the clones over the system or trend charts to evaluate the development of clones over time.
3.)Blacklisting: clones can be blacklisted, so that they do not show up in future detection runs.
Furthermore, ConQAT supports clone detection for additional languages, more specifically C/C++, VB.NET, Java, Cobol and PL/1. (And a bunch of other analyses beyond clone detection, but this is probably out of scope here)
You can find documentation of ConQAT here http://conqat.in.tum.de and on clone detection specifically here: http://conqat.in.tum.de/index.php/CloneDetectionTutorial. We are still lagging behind on documentation somewhat though, so feel free to contact me if you have any questions.
I hope that helps, Best regards,
Elmar
Tuesday, May 20, 2008
Firefox extensions
My laptop went in for repair last week, and whilst it was there having its power cable replaced (internal burning) and screen replaced (deal pixels) I upgrade its hard-drive to be a nice and healthy 120GB. I didn't copy any data across, and after a few hours of re-installs I'm pretty much there, with much thanks to having listed my most used applications on iusethis.
That leaves the firefox extensions though, so I thought I'd list my most used ones here as a record for myself:
If you have some additional extensions that you think are great, let me know and I'll give them a go too
That leaves the firefox extensions though, so I thought I'd list my most used ones here as a record for myself:
- Firebug (obviously)
- Screengrab
- View Source Chart
- TinyURL Creator
If you have some additional extensions that you think are great, let me know and I'll give them a go too
Labels: tool
// posted by Jane @ 3:13 PM
Comments:
Saturday, January 12, 2008
iusethis
I belong to a blogroll of Geek Girl blogs and subscribe to the RSS feed. One post that caught my eye was Véro's post about her Blog Topic Challenge. I left her a comment suggesting "How about a post about your favourite tools for helping you to get stuff done? Or your favourite gadgets? Or your regular RSS reads?".
She has responded with Favourite tools for getting things done sharing her mac apps via the site iusethis. I hadn't spotted this site before, so I've been and set up a profile so that everyone can see what tools I use as well.
She has responded with Favourite tools for getting things done sharing her mac apps via the site iusethis. I hadn't spotted this site before, so I've been and set up a profile so that everyone can see what tools I use as well.
Labels: tool
// posted by Jane @ 10:48 AM
Comments:
Wednesday, August 29, 2007
iGoogle
I started using the iGoogle personalised home page quite some time ago now, and over time have configured it to have almost everything I need displayed on it. This is the screen that I have open on my browser the majority of my day at work, and it gives me a great view into everything that is important.
From top left to bottom right (going in columns) I use:
Every now and again I follow the Add Stuff link to see what is new. And whilst checking that link I've just gone and got distracted by the Ski and Snow Report gadget :-) - one to install in preparation for the big trip me thinks.
From top left to bottom right (going in columns) I use:
- CheckMail - which checks my IMAP email server and lets me know how many new/unread messages I have
- Google Reader - which shows me the title and site name for the most recent 5 (configurable) items
- Weather - configured for Brighton
- Photo of the minute - inspiration for my photography
- Del.icio.us - my most recent 10 (configurable) bookmarks
- Remember The Milk - an interface to Remember the milk, allowing me to add tasks, edit, postpone and complete items on my todo list
Every now and again I follow the Add Stuff link to see what is new. And whilst checking that link I've just gone and got distracted by the Ski and Snow Report gadget :-) - one to install in preparation for the big trip me thinks.
Labels: tool
// posted by Jane @ 6:16 PM
Comments:
Thursday, August 23, 2007
Remember the milk
This time last year I read most of the GTD book and overhauled my existing todo management system (notes in diaries, tasks in outlook, email TODOs etc). After that and up until the beginning of the year I relied on monkeyGTD to help me keep track of what my team and I needed to do. I used it locally, on my desktop and kept backups etc and whilst it worked pretty well for me, it started to get pretty unwieldy due to storing all the tasks within the page itself, not to mention that it was only useful at work and didn't help me when I was at home (there is a password protected online version of monkeyGTD I could have used which would have overcome this issue) so I was still emailing TODO items to my work email address. So, not perfect.
Several months later, with a new (non management) job and tasks building up I decided it was time to move to a new tool. I knew that Paul was a devotee of Remember the Milk (rtm) and so thought I'd give it a try.
First impressions were good, with an easy to use interface allowing me to create as many or as few lists as I needed, with priorities, dates, repeating tasks etc and a handy getting started guide. I discovered an iGoogle gadget which allows an easy method of maintaining a task list. Being an iGoogle fan (there's a blog post coming sometime soon about why this is) this works well for me as the page is open in my browser window quite a lot of the time.
Not long after I started using rtm, they announced twitter integration and being a twitter fan I was totally sold. By linking up twitter to my rtm account, I can:
Having got my twitter direct messages going straight to my mobile phone, I get timely reminders sent to me directly. A fantastic combination of technologies.
Remember the milk have just started to offer a pro service, mainly to offer the chance for regular users to support the service, but there will be some new pro-only features according to the blog post. I've still only been using rtm for two or three months, but I'm already seriously considering supporting them as I'd miss it if it wasn't here, and I've been really impressed with their embracing of technologies (they implemented google gears support within days of it being released). In fact, I'll just add a task to rtm to reminder me to consider that in a months time if I'm still as impressed with it.
Several months later, with a new (non management) job and tasks building up I decided it was time to move to a new tool. I knew that Paul was a devotee of Remember the Milk (rtm) and so thought I'd give it a try.
First impressions were good, with an easy to use interface allowing me to create as many or as few lists as I needed, with priorities, dates, repeating tasks etc and a handy getting started guide. I discovered an iGoogle gadget which allows an easy method of maintaining a task list. Being an iGoogle fan (there's a blog post coming sometime soon about why this is) this works well for me as the page is open in my browser window quite a lot of the time.
Not long after I started using rtm, they announced twitter integration and being a twitter fan I was totally sold. By linking up twitter to my rtm account, I can:
- add tasks to rtm by sending direct messages (d rtm). These get sent to my rtm inbox list. Parsing takes place so if I send a task "Buy milk today @7pm" it'll get added to rtm with an appropriate date and time reminder
- set up my reminder service in rtm to direct message me with tasks due - either daily reminders or the time based reminders
Having got my twitter direct messages going straight to my mobile phone, I get timely reminders sent to me directly. A fantastic combination of technologies.
Remember the milk have just started to offer a pro service, mainly to offer the chance for regular users to support the service, but there will be some new pro-only features according to the blog post. I've still only been using rtm for two or three months, but I'm already seriously considering supporting them as I'd miss it if it wasn't here, and I've been really impressed with their embracing of technologies (they implemented google gears support within days of it being released). In fact, I'll just add a task to rtm to reminder me to consider that in a months time if I'm still as impressed with it.
// posted by Jane @ 6:04 PM
Comments:
