Home Page   We Generate Your Software.
Products Services Company Technology Demo Contact Site Map Search

Full list of in-depth articles - Feedback and questions - Request demo -


Extracting metadata from SQL Server with SoProMach

  1  Mining SQL Server metadata with SoProMach
  2  Further information
  Appendix A  Metadata extracted from AdventureWorks
  Appendix B  Metadata extracted from Northwind
  Appendix C  Metadata extracted from pubs

A previous article discussed a model generation scheme utilizing SoProMach that allows model architects to effectively acquire knowledge and gain control of existing Oracle databases. The very same approach can be used with SQL Server databases, as described in this article. For a full overview and rationale of the suggested approach (CSV vs. XML, XML and Somusar EF files, and so forth) refer to the article on Oracle metadata mining.

1 - Mining SQL Server metadata with SoProMach

The specific solution described in this article allows to extract metadata from Microsoft SQL Server 2005 Express Edition - Community Technology Preview December 2004. The extracted metadata describe the well-known sample databases pubs and Northwind as well as the more recent and feature-rich AdventureWorks database that features SQL schemas.

By means of SoProMach and SQL Server's sqlcmd this solution extracts metadata about the following classes of constructs:

  • tables and constraints;

  • views;

  • stored procedures and functions;

The overall process of this specific solution is shown in the figure below.

Figure 1 - Extracting metadata from SQL Server 2005 Express

Figure 1 - Extracting metadata from SQL Server 2005 Express

The numbers in the figure correspond to the steps described below. Steps 1 to 7 (continuous lines) are project-independent; steps 8-10 (dashed lines) are project-specific.

  1. The Model Architect runs the desired metadata extraction job on the desired database. Note that the Model Architect could be an automated tool;

  2. SoProMach generates on the fly a set of scripts, similar to the queries that a SQL Server DBA would type into a SQL Server's sqlcmd console;

  3. SoProMach submits each script to SQL Server's sqlcmd;

  4. SQL Server's sqlcmd interacts with the RDBMS and retrieves the metadata;

  5. The raw metadata is written by SQL Server's sqlcmd into a set of text files;

  6. SoProMach parses the raw metadata; note that steps 2 to 6 are automatically repeated to fetch metadata about tables, views, stored procedures, and functions;

  7. The interesting part of the metadata is rearranged and stored into CSV files;

  8. CSV files can be examined and analyzed by the Model Architect as plain spreadsheets, or processed by record-oriented tools;

  9. Depending on the project target, the Model Architect runs SoProMach to generate new software, or to produce a new model (XML, or .ef, for instance) for later processing;

  10. According to the target of step #9, SoProMach transforms the uniform metadata into more complete metadata, or directly into software.

2 - Further information

This solution has been tested with Microsoft SQL Server 2005 Express Edition - Community Technology Preview December 2004 and SQL Server's sqlcmd running on both Windows 2000. It has been implemented with SoProMach/Community Edition.

SoProMach/Community Edition and a set of proofs-of-concept are available for free download. All documentation on SoProMach is available on line. For further information please contact us.

Appendix A - Metadata extracted from AdventureWorks

The uniform CSV metadata files produced from database AdventureWorks in step #7 are listed below.

HumanResources.Department.columns.csv
HumanResources.Department.constraints.csv
HumanResources.Employee.columns.csv
HumanResources.Employee.constraints.csv
HumanResources.EmployeeDepartmentHistory.columns.csv
HumanResources.EmployeeDepartmentHistory.constraints.csv
HumanResources.EmployeePayHistory.columns.csv
HumanResources.EmployeePayHistory.constraints.csv
HumanResources.JobCandidate.columns.csv
HumanResources.JobCandidate.constraints.csv
HumanResources.Shift.columns.csv
HumanResources.Shift.constraints.csv
HumanResources.vEmployee.columns.csv
HumanResources.vJobCandidate.columns.csv
Person.Address.columns.csv
Person.Address.constraints.csv
Person.AddressType.columns.csv
Person.AddressType.constraints.csv
Person.Contact.columns.csv
Person.Contact.constraints.csv
Person.ContactType.columns.csv
Person.ContactType.constraints.csv
Person.CountryRegion.columns.csv
Person.CountryRegion.constraints.csv
Person.StateProvince.columns.csv
Person.StateProvince.constraints.csv
Production.BillOfMaterials.columns.csv
Production.BillOfMaterials.constraints.csv
Production.Culture.columns.csv
Production.Culture.constraints.csv
Production.Document.columns.csv
Production.Document.constraints.csv
Production.Illustration.columns.csv
Production.Illustration.constraints.csv
Production.Location.columns.csv
Production.Location.constraints.csv
Production.Product.columns.csv
Production.Product.constraints.csv
Production.ProductCategory.columns.csv
Production.ProductCategory.constraints.csv
Production.ProductCostHistory.columns.csv
Production.ProductCostHistory.constraints.csv
Production.ProductDescription.columns.csv
Production.ProductDescription.constraints.csv
Production.ProductDocument.columns.csv
Production.ProductDocument.constraints.csv
Production.ProductInventory.columns.csv
Production.ProductInventory.constraints.csv
Production.ProductListPriceHistory.columns.csv
Production.ProductListPriceHistory.constraints.csv
Production.ProductModel.columns.csv
Production.ProductModel.constraints.csv
Production.ProductModelIllustration.columns.csv
Production.ProductModelIllustration.constraints.csv
Production.ProductModelProductDescriptionCulture.columns.csv
Production.ProductModelProductDescriptionCulture.constraints.csv
Production.ProductPhoto.columns.csv
Production.ProductPhoto.constraints.csv
Production.ProductProductPhoto.columns.csv
Production.ProductProductPhoto.constraints.csv
Production.ProductReview.columns.csv
Production.ProductReview.constraints.csv
Production.ProductSubcategory.columns.csv
Production.ProductSubcategory.constraints.csv
Production.ScrapReason.columns.csv
Production.ScrapReason.constraints.csv
Production.TransactionHistory.columns.csv
Production.TransactionHistory.constraints.csv
Production.TransactionHistoryArchive.columns.csv
Production.TransactionHistoryArchive.constraints.csv
Production.UnitMeasure.columns.csv
Production.UnitMeasure.constraints.csv
Production.WorkOrder.columns.csv
Production.WorkOrder.constraints.csv
Production.WorkOrderRouting.columns.csv
Production.WorkOrderRouting.constraints.csv
Production.vProductAndDescription.columns.csv
Purchasing.ProductVendor.columns.csv
Purchasing.ProductVendor.constraints.csv
Purchasing.PurchaseOrderDetail.columns.csv
Purchasing.PurchaseOrderDetail.constraints.csv
Purchasing.PurchaseOrderHeader.columns.csv
Purchasing.PurchaseOrderHeader.constraints.csv
Purchasing.ShipMethod.columns.csv
Purchasing.ShipMethod.constraints.csv
Purchasing.Vendor.columns.csv
Purchasing.Vendor.constraints.csv
Purchasing.VendorAddress.columns.csv
Purchasing.VendorAddress.constraints.csv
Purchasing.VendorContact.columns.csv
Purchasing.VendorContact.constraints.csv
Purchasing.vVendor.columns.csv
Sales.ContactCreditCard.columns.csv
Sales.ContactCreditCard.constraints.csv
Sales.CountryCurrency.columns.csv
Sales.CountryCurrency.constraints.csv
Sales.CreditCard.columns.csv
Sales.CreditCard.constraints.csv
Sales.Currency.columns.csv
Sales.Currency.constraints.csv
Sales.CurrencyRate.columns.csv
Sales.CurrencyRate.constraints.csv
Sales.Customer.columns.csv
Sales.Customer.constraints.csv
Sales.CustomerAddress.columns.csv
Sales.CustomerAddress.constraints.csv
Sales.Individual.columns.csv
Sales.Individual.constraints.csv
Sales.SalesOrderDetail.columns.csv
Sales.SalesOrderDetail.constraints.csv
Sales.SalesOrderHeader.columns.csv
Sales.SalesOrderHeader.constraints.csv
Sales.SalesOrderHeaderSalesReason.columns.csv
Sales.SalesOrderHeaderSalesReason.constraints.csv
Sales.SalesPerson.columns.csv
Sales.SalesPerson.constraints.csv
Sales.SalesPersonQuotaHistory.columns.csv
Sales.SalesPersonQuotaHistory.constraints.csv
Sales.SalesReason.columns.csv
Sales.SalesReason.constraints.csv
Sales.SalesTaxRate.columns.csv
Sales.SalesTaxRate.constraints.csv
Sales.SalesTerritory.columns.csv
Sales.SalesTerritory.constraints.csv
Sales.SalesTerritoryHistory.columns.csv
Sales.SalesTerritoryHistory.constraints.csv
Sales.ShoppingCartItem.columns.csv
Sales.ShoppingCartItem.constraints.csv
Sales.SpecialOffer.columns.csv
Sales.SpecialOffer.constraints.csv
Sales.SpecialOfferProduct.columns.csv
Sales.SpecialOfferProduct.constraints.csv
Sales.Store.columns.csv
Sales.Store.constraints.csv
Sales.StoreContact.columns.csv
Sales.StoreContact.constraints.csv
Sales.vIndividual.columns.csv
Sales.vSalesPerson.columns.csv
Sales.vStore.columns.csv
dbo.AWBuildVersion.columns.csv
dbo.AWBuildVersion.constraints.csv
dbo.ufnGetAccountingEndDate.signature.csv
dbo.ufnGetAccountingStartDate.signature.csv
dbo.ufnGetStock.signature.csv
dbo.uspGetBillOfMaterials.signature.csv
dbo.uspGetEmployeeManagers.signature.csv
dbo.uspGetManagerEmployee.signature.csv
dbo.uspGetWhereUsedProductID.signature.csv

Appendix B - Metadata extracted from Northwind

The uniform CSV metadata files produced from database Northwind in step #7 are listed below.

dbo.Alphabetical list of products.columns.csv
dbo.Categories.columns.csv
dbo.Categories.constraints.csv
dbo.Category Sales for 1997.columns.csv
dbo.Current Product List.columns.csv
dbo.CustOrderHist.signature.csv
dbo.CustOrdersDetail.signature.csv
dbo.CustOrdersOrders.signature.csv
dbo.Customer and Suppliers by City.columns.csv
dbo.CustomerCustomerDemo.columns.csv
dbo.CustomerCustomerDemo.constraints.csv
dbo.CustomerDemographics.columns.csv
dbo.CustomerDemographics.constraints.csv
dbo.Customers.columns.csv
dbo.Customers.constraints.csv
dbo.Employee Sales by Country.signature.csv
dbo.EmployeeTerritories.columns.csv
dbo.EmployeeTerritories.constraints.csv
dbo.Employees.columns.csv
dbo.Employees.constraints.csv
dbo.Invoices.columns.csv
dbo.Order Details Extended.columns.csv
dbo.Order Details.columns.csv
dbo.Order Details.constraints.csv
dbo.Order Subtotals.columns.csv
dbo.Orders Qry.columns.csv
dbo.Orders.columns.csv
dbo.Orders.constraints.csv
dbo.Product Sales for 1997.columns.csv
dbo.Products Above Average Price.columns.csv
dbo.Products by Category.columns.csv
dbo.Products.columns.csv
dbo.Products.constraints.csv
dbo.Quarterly Orders.columns.csv
dbo.Region.columns.csv
dbo.Region.constraints.csv
dbo.Sales Totals by Amount.columns.csv
dbo.Sales by Category.columns.csv
dbo.Sales by Year.signature.csv
dbo.SalesByCategory.signature.csv
dbo.Shippers.columns.csv
dbo.Shippers.constraints.csv
dbo.Summary of Sales by Quarter.columns.csv
dbo.Summary of Sales by Year.columns.csv
dbo.Suppliers.columns.csv
dbo.Suppliers.constraints.csv
dbo.Ten Most Expensive Products.signature.csv
dbo.Territories.columns.csv
dbo.Territories.constraints.csv

Appendix C - Metadata extracted from pubs

The uniform CSV metadata files produced from database pubs in step #7 are listed below.

dbo.authors.columns.csv
dbo.authors.constraints.csv
dbo.byroyalty.signature.csv
dbo.discounts.columns.csv
dbo.discounts.constraints.csv
dbo.employee.columns.csv
dbo.employee.constraints.csv
dbo.jobs.columns.csv
dbo.jobs.constraints.csv
dbo.pub_info.columns.csv
dbo.pub_info.constraints.csv
dbo.publishers.columns.csv
dbo.publishers.constraints.csv
dbo.reptq1.signature.csv
dbo.reptq2.signature.csv
dbo.reptq3.signature.csv
dbo.roysched.columns.csv
dbo.roysched.constraints.csv
dbo.sales.columns.csv
dbo.sales.constraints.csv
dbo.stores.columns.csv
dbo.stores.constraints.csv
dbo.titleauthor.columns.csv
dbo.titleauthor.constraints.csv
dbo.titles.columns.csv
dbo.titles.constraints.csv
dbo.titleview.columns.csv


Written on 10 January 2005 - Updated on 6 September 2005
Full list of in-depth articles - Feedback and questions - Request demo -

http:// www.somusar.com  / company  / news  / in_depth  / sql_meta  - Powered by SoProMach
Copyright © 2003-2012 Somusar - Trademarks - Legal - Privacy - Webmaster