Author Archive

Introducing SuperVIEW Collaboration

Wednesday, February 3rd, 2010 by Jo Deeker

SuperVIEW is our solution for Interactive Publication, Exploration & Visualization of Public Data. Our latest version has a new collaboration feature that we want to share with you.

Using our new SuperVIEW Collaboration features, you can make comments or invite others to make comments on your visualizations using Google Friend Connect.  You can also share your customized visualisation with others using our new Share feature. The Share feature allows you to embed a link to your view in a website, blog, Facebook, Twitter or your other favorite social networking application.

Recently Craig Thomler, a well-known active participant and leader in the Australian Gov2.0 movement, wrote a blog post on the new data.gov.uk site which he considers is the world leader in open data websites.  He then goes on to make a wishlist of what we could do in Australia to the data.australia.gov.au site to make it the best in the world.  Some of what he is asking is for is delivered by SuperVIEW right now including the ability for people to embed visualizations into their own sites, and to allow every set of data to support a discussion to allow people to ask questions to clarify what the dataset contains and discuss how it could be presented in a more usable way.

View this video to see SuperVIEW Collaboration in action.

If you have any questions about SuperVIEW please contact  jo.deeker@spacetimeresearch.com

SuperVIEW Version 1.4

Friday, December 4th, 2009 by Jo Deeker

Every month we release a new build of SuperVIEW and the team behind the development are Agile masters. Each build contains new and improved features for data geeks, new visualizations, and of course fixes for the bugs… we even go on safaris to find them.

Hybrid Cloud Service

The SuperVIEW Hybrid Cloud Service consists of two components:

  • The SuperVIEW Web application in a cloud service provided by the Google App Engine.
  • The application is connected to the ‘back-end’ SuperSTAR server that cross-tabulates and processes the data.

Learn more about the Hybrid Cloud Service ….

Showcase Visualizations

Top-N charts

Top-N charts sort and filter datasets to provide an easy visual comparison of relative data event sizes. They allow you to integrate very large classifications into SuperVIEW sites by filtering in only the Top-N items in a given query. For example the top 10 locations out of 100.

Top N Chart

Population Pyramid

The Population Pyramid enables a visualization of demographic trends through population pyramids that stack two distributions back to back and side by side.

Population pyramid

Previews of New Visualizations

Dual-axis Chart

The Dual Axis Chart plots two data series against each other.

Dual-axis Chart

Timeline Chart

The Timeline Chart is based on the Google Visualization API. This chart allows you to select a time period from a scale at the base of a chart, and then see the data updated and stretched to fit the width of the screen. You can also zoom in and out of a time range.

time-line-chart

Side-by-Side Pie Charts

This allows you to view two pie charts side-by-side.

Side-by-side Pie Charts

Features for Data Geeks

Dynamic Recodes

The configuration of the dynamic recodes feature used in the Data Selection Experience has been streamlined. You now can multi-select or de-select filters.

Want more

Contact Space-Time Research if you want more information or leave us a comment in this blog post.

How Safe Is Your Suburb - Mashup entry

Friday, November 13th, 2009 by Jo Deeker

How Safe Is Your Suburb was an entry in the Mashup Australia contest.

Click here to try How Safe Is Your Suburb

How Safe Is Your Suburb is an easy-to-use interactive web application for the public to gain greater insight into crime statistics in Local Government Areas (suburbs) in New South Wales. The application can be used for informed discussion and policy development by residents, police authorities, and local government. The applicaion shows how statistics can be applied in the everyday life of the community.

How Safe Is Your Suburb embraces the Gov 2.0 philosophy by opening up a static dataset to the public in a useful way.  The user can analyse and play with the data, comment on data, and then share their data with others.

For example, the user can choose different ‘reports’, make selections within each report to compare different types of crime over time, and then see which types of crime are more prevalent in their area.  They can view an interactive thematic map of crime that provides a spatial visualisation of crime types across LGAs for a given year.  They can also identify which suburbs have higher crime rates in total and in per-head of population. (It makes sense that there is more crime in more populous areas).  Users can make comments on each visualisation they are working on.

The application mashes up NSW crime data with LGA boundary files and Census data from ABS.  Space-Time Research has classified each offence into different categories to enable simpler analysis.  More detail could be added to the application at a later date.

The application is built using Space-Time Research’s SuperVIEW product, and is hosted on the Google App Engine. In the spirit of a govhack style competition, our team of three (one database builder, one programmer, one analyst / writer) started working on the application just over 24 hours before it was due.

We would also like to share our experience of mashing up and visualising the data. We have found:

  • There is an unexpected spike in road traffic offences in 2001 and 2002 and then no road traffic offences recorded after that.  This is seen across most LGAs. Only by visualizing the data in a chart did we see the problem, and would suggest that the data quality be checked with NSW Bureau of Crime Statistics and Research before releasing this data. Perhaps the data should be footnoted.
  • We discovered gaps when joining by LGA – our map file, the ABS census data and the NSW Bureau of Crime Statistics and Research data all have slightly different datasets.  We don’t know what year the LGAs in the source data were referenced to, and our application currently joins on LGA name rather than LGA id.
  • We chose to refer to the spatial areas as ‘suburbs’ to make it easier for the general public to relate to. We are aware that LGAs are different from postcode boundaries and that the general public will not be aware of the difference between the two types of geographic boundary. Most members of the public may not know what an LGA is and we have referenced suburbs with LGA in parenthesis throughout the application.

Ideas for enhancing the application include:

  • Enhancing the share functionality by including a share this on twitter, facebook etc application.
  • Expanding the application to allow analysis by individual offence types.
  • Incorporating other ABS census demographic data, such as population count to calculate offences per head of population, and inclusion of employment, education, age breakdown etc. to see if demographics of an LGA impact crime rates.

Why APIs are important for Gov2.0

Wednesday, October 21st, 2009 by Jo Deeker

I was at the Gov 2.0 conference in Canberra earlier in the week and found that compared to the talk around social engagement through Twitter and Facebook, the whole concept of open data and APIs took a back seat for much of the event. APIs were mentioned by speakers, but I did not get any sense that the majority of the attendees were thinking about APIs and mash-up-ability of data as much as I do. I also wasn’t sure that everyone knew what an API was, or why you would want one.

So we asked our Director of Product Planning, Don McIntosh to write an article about what APIs are, and why they’re important. This is what he has to say about APIs.

With social applications, there is a clear and obvious use that everyone can understand, and the staggering traffic volumes for these sites make the topic all the more compelling. But what about open data and APIs? Why should we pay them any attention and how do we benefit from them?

An API is an Application Programming Interface. Web based APIs, sometimes referred to as Web services, are growing at a phenomenal rate. Basically, instead of information being presented in a predetermined manner through Web pages, APIs allow other applications (iPhone apps, Websites, MS Windows applications….) to extract specific chunks of information and combine it with other information in all kinds of ways to serve a specific purpose. Jim Ericson from Information Management blogged about this, and he included a good description of how Web services get used:

“Now think of all the thousands of iPhone apps and how they amalgamate all kinds of Web services. You open your commuter traffic app, it calls on traffic information services, Google maps, a weather forecast and maybe an ad for public transportation. One browser app, many (API) calls.”

Jim also mentioned how prominent APIs are becoming. For many popular websites, the network traffic generated by APIs actually exceeds the direct Web traffic. And that’s expected to continue. Perhaps even more interesting is the fact that these days, you don’t even need to be a programmer to use Web APIs. If you have played with Yahoo Pipes, or similar mashup tools, you know what I mean. Basically, these tools are empowering end users to create their own custom applications. Just drag and drop – no coding required.
So, they’re useful, widely used, accessible even to non-programming types, and becoming more popular by the day but what in particular makes them so important in a Gov 2.0 context? I’d summarise it by saying that it’s about making it possible (and easy) for those outside of government to present statistics in a context that is meaningful and useful for them, and that can help facilitate informed discussion and decision making. If I want to provide a service to help people decide where to live, I could combine census statistics such as occupation, income, and age and mash it up with information about the location of shopping centres, pubs etc from a different service. I could achieve the same by gathering all the data into a database and building my service on top, but by accessing the data through an API, my information can remain current, and my queries can be run by calls to the API, saving me from the complexities and resources required to process the data myself. I can also leverage other services such as Google maps to present results. And of course, thanks to mashup platforms, this kind of application might just be something that an (non-programmer) individual does to satisfy their own interest. Either way, it makes it much more possible for people to take government information and use it in ways that government may never have chosen to do.

From a data provider’s perspective, there are many things to consider when looking at providing APIs for direct data access and querying.

1. API vs other means

An API can facilitate innovation, and help automate services that other organizations may provide based on the data. It can also provide transparency by not colouring the data in any particular way, but leaving it open to others to render analysis of the data in their own way. On the other hand, if representing the data in certain ways is useful in promoting an organization’s mission, then it might be best to concentrate on delivering the appropriate views and/or viewing tools for the data. Or in some cases, it might make sense to do both.

2. Risk of abuse

Gartner analyst Andrea diMaio noted that separating data from its source and having no clear way to let consumers understand its lineage or quality runs a great risk of it being misused, or deliberately doctored to represent the “facts” that best suit the application builder. What does this mean to the organization providing the data? Providers of official statistics go to great lengths to defend against this possibility yet by providing data through APIs, they may in some way increase the risk of this happening. Perhaps one way to look at it is to realise that this can happen anyway, without APIs. And it is probably unreasonable to expect a provider to do more than provide accurate quality information alongside their data (and even make it queryable through the API) so that users can make informed choices about what constitutes valid use of the data.

3. Data privacy protection

Many statistical agencies have “remote access data laboratory” services to give researchers the ability to perform detailed analyses on their data. There are typically manual checking processes in this, to ensure that researchers’ queries do not breach data privacy laws by identifying individuals from the data (something that is very easy to do, even when data has been anonymized). A provider would need to determine what privacy risks are posed by making the data available through an API, and ensure that appropriate safeguards are put in place.

4. Resources

An API call results in some amount of processing. Depending on the specifics, such as the type of query and the volume of data, the level of computing resources required can be quite significant. In the beginning, one option may be to limit API use to a few specific applications, and expand that over time. Alternatively, the API could impose certain limits for any single user. This is the approach that Twitter uses to manage the enormous demand it generates.

SuperSTAR Goodies - 6.7 Release progress

Tuesday, October 13th, 2009 by Jo Deeker

We would like to share the progress of some of the good stuff we have been doing in SuperSTAR development towards our 6.7 release.

Since transitioning to a fully agile process, we now run fortnightly iterations. From time to time, we will share the outcomes of an iteration and keep you all up to date.

Some of the key items that came out of this iteration were:

1. Record View in SuperWEB2 - we have implemented our first two user stories:
“As a SW RecordVIEW user, I want a way of seeing all the unit records that relate to a crosstab table so that I can understand the detail behind the crosstabulation”.
“As a SW RecordVIEW user, I want filtered view of the unit records that relate to the cells in a crosstab table I choose so that I can focus on specific areas of interest”

We have implemented RecordView using GWT in the RESTful style. GWT allows us to get a Rich Internet client user experience. Using REST means that it is easy for other clients such as SuperView to consume the RecordView service.

2. Aggregated mapping for SuperWEB2
“As a SW2 user, I want to have a faster mapping experience so that I can be more productive”.

The Mapping team have done some great work to improve the performance of our mapping solution in SuperWEB2. They have developed a ArcGISMap widget which allows SuperWEB2 to communicate directly with the Arc GIS Server via a REST interface. This means much faster zoom and pan performance with maps.

3. SuperCROSS Local Annotations Refactor – we are making good progress to get the Annotations working correctly again in SuperCROSS and are on track with our plans.

4. Automated testing – we have also made good progress in automating the testing of SuperCROSS and SuperWEB2.

If you have any questions regarding our progress on the 6.7 release, or about any SuperSTAR product, please do not hesitate to contact us at support@spacetimeresearch.com

Record VIEW Functionality in SuperWEB2 - comments welcomed

Sunday, October 4th, 2009 by Jo Deeker
Record View

Record View

A guest blog from Don McIntosh, our product manager for SuperSTAR. Please feel free to give us comments or feedback so we can incorporate your feedback into our product development while we are developing it.

What I wanted to cover in this post is a brief summary of what we are planning for RecordVIEW, as well as a few features that might come in a later release. I wanted to write about this now while we are developing it so that our customers and partners have an opportunity to comment and hopefully improve on the end result. Another thing we’ll do is provide a link to a test instance to let you play around with it once we have it up and running.

RecordVIEW is a key feature of SuperWEB - and one that is currently lacking in SuperWEB2. It gives users the ability to drill down into the records that contribute to any cell in a table and view other attributes of those records. We find that customers use it for a variety of reasons. Two of the most common reasons are identification of individuals in interesting sub-populations, and data validation. An example of the former is “give me the list of names of all students scored above 95% in the English test”. An interesting point is that almost all the time, the records extracted via

RecordVIEW need to be subsequently fed into another system for the user to complete their task. That’s a useful one for us to keep in mind, because perhaps we can add much more value by allowing some kind of direct integration between the RecordVIEW action and other systems.

The first step for RecordVIEW is actually to cover off much of the functionality we had in the original SuperWEB. That means identifying some cells, switching to the RecordVIEW tab, choosing what fields to report on, and then downloading to XLS or CSV. The major addition for the first release in comparison to what was in the original SuperWEB will be in the ease of use. The experience will be a lot more immersive, with fewer pauses for server updates and a richer UI. Click on a cell, chose RecordVIEW and then choose what fields to view. You can choose all fields, or start with none and add a select few. You can also sort the results, and selectively filter what fields you’re interested in viewing. One other key feature I’ll mention is that the results of the RecrordVIEW are transparently paginated, so if you have a very long list, the browser isn’t waiting a long time to update it; it simply adds more as you scroll down.

We are of course very aware that for some datasets, RecordVIEW is not appropriate, due to the sensitive nature of the data. We will keep this simple: if there is confidentiality enabled for a database, then no RecordVIEW. Other permission functionality will remain unchanged from the earlier version.

Other key features we will consider later on include cell selection from other views, such as areas on a chart or map. Also, as I mentioned earlier on, we’d like to explore how we can get RecordVIEW output might be more tightly integrated a workflow that involves taking sets of records to feed into another application for further processing, or viewing in a certain way.

It would be interesting to hear about some usage scenarios of feature ideas for RecordVIEW from our customers. We may be able to incorporate some scenarios in our acceptance testing, and hopefully learn about some ways to make this feature smarter and more in line with users’ core needs.

RecordVIEW will be available the Release 6.5 November service pack.

We’re in the cloud! SuperWEB available now

Thursday, October 1st, 2009 by Jo Deeker

I’m really excited to announce that we aim to be among the first companies to host applications on the Apps.gov website.

To get there, we needed to get SuperWEB up into the cloud, and this week, we hosted our first application on the Amazon EC2 cloud. Yesterday, I got my first Amazon bill - $10 / day so far and we uploaded a lot of data!

Background:

Vivek Kundra, the US Federal Chief Information Officer, has launched the new Apps.gov Storefront to enable US Federal Government agencies to buy cloud computing services as easily as a consumer can acquire a Gmail or Facebook account.

Cloud computing services reduce costs through reductions in purchasing and maintaining servers, while simultaneously improving service scalabilty to manage peaks and troughs in usage. Kundra says that besides encouraging better collaboration among agencies, he expects cloud services to reduce energy consumption because agencies will be able to share IT infrastructures.

Space-Time Research is responding to the recent US Federal Government request for proposal for applications to be hosted via the Apps.gov website. The Apps.gov Storefront is managed by the US GSA (General Services Administration) and SuperSTAR software is already available for purchase through the GSA e-Library.

Space-Time Research cloud offerings

In September, Space-Time Research initiated a cloud offering by hosting SuperWEB Software as a Service (SaaS) on the Amazon EC2 cloud service. SuperWEB is currently in the process of being assessed for inclusion in the Apps.gov website. Once certified, SuperWEB SaaS will be available to buy as a small, medium, large or extra large implementation on a pay-by-month basis.

At the end of October, SuperVIEW will be production-ready and available via a Google App Engine hybrid cloud service. For more information, see SuperVIEW hybrid cloud service.

More about Apps.gov

Apps.gov is managed by the GSA development team, which is led by Casey Coleman, GSA’s CIO. In the article Kundra’s great experiment: Government apps ’store front’ opens for business, Coleman says:

“Through Apps.gov, GSA can take on more of the procurement processes upfront, helping agencies to better fulfill their missions by implementing solutions more rapidly,”

“We will also work with industry to ensure cloud-based solutions are secure and compliant to increase efficiency by reducing duplication of security processes throughout government.”

Jo Deeker

My favourite sites

Tuesday, September 22nd, 2009 by Jo Deeker

My three favourite sites at the moment are:

As we enable public intelligence and data provision, and we’re an Australian based company, I have to keep on top of this every day. I love how fast ideas are moving.
http://gov2.net.au

For all goodness in quality and testing management. If I ever have a question or problem to solve and I’m stuck, I go here. Good for inspiration and great ideas.
https://www.stickyminds.com

Just launched by the US Government and we’re going to be on it soon with a cloud provision of SuperWEB. Any US Government Agency will be able to buy us through this process. Super-excited about this one.
http://apps.gov

Jo

Bug Safaris - a different way to find bugs

Thursday, September 17th, 2009 by Jo Deeker

Here is a post from Adrian Mirabelli - a Customer Quality engineer at Space-Time Research. The idea for a bug safari came out of a presentation at the ANZ Test Board Conference in March 2009.

Bug safaris at Space-Time Research

For release 6.5, the STR quality team introduced “bug safaris” as a way to effectively and quickly find software bugs.

A bug safari involves the majority of the organisation including development, design, and management to locate bugs. Test cases or scripts are not necessarily provided but guidance should be given. Certain areas are targeted and the amount of interruptions is minimised to increase the effectiveness. Note the bug safari can be held multiple times over a release.

Planning is the key!

At the beginning of a bug safari, the quality manager invites the participants to a planning session or “kickoff”. The purpose of the kickoff is to define:

  • The objectives of the session – including to communicate what is being done; all participants should be very clear about this by the conclusion of the session
  • Areas to test and who will do it – this is important to ensure coverage and no wasteful duplication
  • Configuration required and who will do it
  • Test cases or documentation required and who will do it – the structure of these products should be agreed, for example, is it a checklist or a matrix that is filled out on-the-fly?
  • Some ideas of how to test – do something unusual or non-typical, test boundary values, do something unusual
  • Duration of session
  • Method for reporting issues and bugs

Typically the system configuration and documentation will be done by the testing team with the help of technical resources if required. Login information is distributed in advance. The quality manager needs to decide how to report results including submission of bug reports, and therefore plays a crucial role in this testing.

At the agreed date or time, the testing itself is performed, typically no longer than two hours but longer than 45 minutes. This session is generally intense in nature as the mission is to find problems. The system testers are usually assigned to a product and work with the participants to help identify issues and troubleshoot problems. They can also be actively testing the system depending on what is agreed at the kickoff session.

At the conclusion of the session, results are tabulated and any bugs found are raised in the incident tracking system.

Within the next couple of days at the absolute latest, a debriefing is held with the participants including the system testers. The quality manager reports on bugs found, and discussion is held regarding:

  • The perceived level of success of the bug safari
  • What can be improved for next time
  • What worked well this time
  • General feelings and sentiments
  • Required actions and action owners.

Why not just use structured tests?

Procedural test cases, which follow a step-by-step test script, are excellent for communicating to the wider audience how you are testing and to obtain buy-in and feedback from stakeholders. In my experience, however, you can find bugs by looking around the software, not just looking at the expected results of the test case. Further, bugs are found when testing certain sequences of data, mouse clicks, configuration, operating systems and more, and it is expensive to write test cases for all these combinations.

Why involve people outside the testing team?

You and I are testing software every day. Just by using software you are testing whether it satisfies your need and your purpose. Everyone interacts with software differently, and is likely to try things out in various and different ways, some typical and some strange, so it is good to have such testing sessions to really verify the software is “fit-for-purpose”. It also gives the opportunity for fresh eyes to look and question the software, and test out other important elements such as usability and compatibility. It also increases the participants’ knowledge of the software, whilst testing the accuracy of the configuration and documentation, including the quality of test harnesses and pre-defined scripts.

What are the benefits?

Bug safaris are defined as “exploratory testing” with more tangible results. The results can easily be reported on charts or whiteboards and transferred to the test management and tracking system.

We allow the participants to exercise freedom of thought in executing tests. In this way we can find new bugs as possible new combinations of tests are being exercised. Quality therefore improves as we can address and fix such bugs based on their priority. The participant is encouraged to investigate and should investigate any strange behaviour they find, perform further tests, and ask questions.

By everyone being involved in testing, and not just the test team, it improves the visibility of the test organisation and the importance of testing, whilst sharing the ownership of “quality” to all people involved in the development of the software from concept to implementation.

By performing such tests, we can report and therefore utilise many metrics to find out, for example:

  1. Number of bugs or issues found per session
  2. Number of sessions run
  3. Areas covered in the session with combinations
  4. Time required to configure
  5. Time required to test
  6. Time required to investigate issues

The key is that people work together and discuss openly the software and what it does.

What are the challenges?

This method of testing is still relatively new, and is therefore not a perfect method, nor a substitute for traditional testing methods. The key is to balance out the proportion of how much testing is structured versus unstructured, whilst ensuring that the testing results are captured sufficiently. Such examples of test tracking may require the participant to complete a spreadsheet, matrix or running sheet.

What is being done in future?

Space-Time Research will run bug safaris in future releases. Bug safaris have been shown to find bugs, and important bugs, and are continuing to win flavour in the testing industry as its true benefits are being realised. Introducing bug safaris have the advantage of not requiring major cultural or system changes, or expensive start-up costs.

Our Quality Vision (and Addressing Our Quality Past)

Monday, August 24th, 2009 by Jo Deeker

Like all software companies, we at Space-Time Research have juggled customer demands, complex software, very different uses of our software, and ever changing requirements. This has sometimes resulted in us delivering release software to our customers that is not of a sufficient quality, and later than we planned.

In the past, and as recently as our 6.3 release of our software, our testing group has passed a release and the software has been delivered to a customer and then a critical issue has been found. One of the main reasons this happens is that every customer has a slightly different environment. We currently support Solaris, Red Hat Linux, Windows 64 bit, Windows 32 bit, Windows XP and Vista for our client applications, browsers including IE6, IE7, IE8, Chrome, Firefox, Safari. We read data from any relational database that has a jdbc driver including Oracle, SQL Server, DB2 and others, plus different types of text files. We provide mapping with ESRI ArcIMS, ArcGIS Server, Google Maps and soon Bing Maps. We test all these environments and on our servers, our testing can pass.

Then we get out to the customer environment and encounter different environments & constraints. Not everyone can host a Tomcat application and we might have to hook to IIS. Firewalls might be an issue. Ports might be an issue. The client might operate in a remote way. Even if we don’t officially support a configuration, our clients will implement that way anyway and it’s up to us to sort it out.

Once we have the software successfully installed and configured at a client site, they then build some databases and work out how they are going to analyse or visualise their information. Every client has different types of databases, structures and uses of their information. Our testing doesn’t cover every different type of database - we try to, but of course we don’t cover everything. So sometimes we miss things - heirarchical summation options being a recent example.

Finally, our customers use the software with their own workflow. We follow a standard workflow with our automated tests, and then we conduct exploratory testing that mimics what a customer would do, but as we are not the customer, we don’t always get that exactly right either.

So, how do we improve it? What have we done and what are we doing next?

Firstly, for our 6.5 General Availability Release, Space-Time Research defined the following quality vision:

  • Timely, relevant, functioning software that works!
  • Performance, stability and resiliency focus.
  • Deliver releases of SuperSTAR that are perceived within STR and by our partners and customers as better than the previous release.

All decisions about testing, and then which bugs we fix, and when we release our software, are related back to the quality vision.

We implemented a partnership approach with some selected customers to enable them to test pre-release versions of our software. We conducted fortnightly builds, ran a couple of days of testing and then made the builds available to the customer. Builds were provided via FTP site, and customers were able to download the software and install in their own test environments. The customers were able to choose whether they would take a build or not. STR also hosted versions of our web applications so customers could do user interface testing without having to run their own installation and configuration.

The customers reported bugs, severity and their own priority via our normal support channel (via email to support@spacetimeresearch.com). We regularly triaged the bugs reported, and communicated via conference call with each customer to advise what we intended to do, or discuss concerns.

The benefits of this approach were clear for each customer involved:

  • Integration and configuration issues were ironed out during the pre-release phase.
  • Customer-focused testing found issues we would never have found.
  • The end delivery held no surprises.
  • We delivered on time to those customers and met their deadlines.

6.5 General Availability release is almost complete on all platforms. I’ll do another blog and announcement about that separately.

For our next release, we are implementing a fully agile development process. Another blog on that is coming too! But for our customers, please know that we want to:

  • Involve more customers in pre-release testing.
  • Collect more sample databases from customers.
  • Collect reference data sets from customers so we can validate our statistical routines.
  • Use client test beds for complex or unusual environments.
  • Open up our change management and support processes so customers can track issues they are interested in.

Cheerio

Jo