Showing posts with label data science. Show all posts
Showing posts with label data science. Show all posts

Friday, May 17, 2024

Using SQL Server with Python

I am starting a brief series of posts on manipulating a SQL Server database with Python. This is a beginner level tutorial for students wanting to get started with this activity. Python provides access to a large ecosystem of data science functions above the normal SQL Query results. We will eventually work our way up to these. But first we will start with the basics: creating an ODBC connection to a SQL Server database followed by a series of posts on the standard CRUD (Create, Read, Update, Delete) functions.

These examples were done using Windows 10. To follow along you will need to have Python installed (we are using Python 3) as well as SQL Server (it should not matter too much which version so long as it is fairly recent. We are using v16). The tutorial uses a build script which we have supplied on our Google drive. This creates a sample database called bookstore. Install this and then create an ODBC data source using the Windows ODBC data sources applet. If need be, download and install the odbc driver 17 from Microsoft first.

To connect your python scripts to SQL Server you need one more component, the Python package to connect to an ODBC data source: pyodbc. You may find you already have it installed but if not you can just install pyodbc using pip in the command window.

>pip install pyodbc

With the above in place you can adjust and run the following Python script to connect to the database. Modify the server to equal <your computer name>\<SQL Server instance name>. 

import pyodbc
# assumes bookstore database has been created in sql server
def open_database():
    """ Open connection to bookstore database using ODBC"""
    cnxn = pyodbc.connect('Driver={ODBC Driver 17 for SQL Server};'
                      'Server=HP10ALL-IN-1\SQLEXPRAS2016A1;'
                      'Database=bookstore;'
                      'Trusted_Connection=yes;')
    print('\nConnected to SQL Server with ODBC');
    return cnxn  
# main program    
cnxn = open_database()
cnxn.close()
print("Database connection closed")

I have used Windows Authentication but you can use SQL Server authentication if you prefer. You just need to supply the user name and password in lieu of Trusted_Conection=yes.

Once this is working you can proceed to the next steps which we will provide in following posts: 1) read data, 2) add records, 3) update data, 4) delete records. After that we will extend the series to include DDL - creating and dropping databases and tables, etc. We will also cover creating and using a stored procedure.

Monday, May 1, 2017

Here is a summary of the years that various programming languages appeared. Again drawn from Wikipedia. I was looking for the release dates for R and other languages and software packages used in Data Science and decided to generalize the searches.


Friday, April 28, 2017

Popularity of some programming languages/IT skills

What follows is a little survey I did on indeed.com on 4/28/17. I started off just looking for the popularity of the R language for data science but wound up broadening the search terms. I limited the search to my home state of California. It is not in  any way a scientific survey but it did prove interesting as far as it went. R ranked pretty high. Right behind Python and ahead of C++. C and SQL were the top two languages. Java was 3rd. We already offer a short coding class in Java and are developing one in Python. Perhaps we will do R next. A couple other observations are that the open languages are firmly in the lead. SAS had the most mentions of the commercial statistical packages.

I often say that SQL is a foundation skill for data analysis. This survey seems to bear that out. To that I would add Python and/or R for those interested in a data science career.


skill/termindeed mentions
microsoft excel28,420
c23,962
sql15,825
linux14,165
java13,769
python12,835
r9,826
c++8,771
javascript8,619
microsoft access8,199
bi7,778
hadoop4,461
ruby3,715
sas3,602
mysql3,601
tableau3,413
business intelligence3,390
spark3,284
rails2,677
epic2,618
matlab2,141
etl1,726
scala1,685
django668
oracle542
spss538
minitab273
stata236
cobol135
fortran132
medisoft20
statistica7

Wednesday, April 26, 2017

La Palma STEM Symposium

Here are the slides for a presentation I gave on Big Data and Data Science at the La Palma, California STEM Symposium. This is a booming field for people interested in data analysis.

Wednesday, April 19, 2017

La Palma STEM Symposium

FYI - I will be presenting a brief talk on Big Data and Data Science at this symposium. It is mainly aimed at Community college students and the local community. See link below.

La Palma STEM Symposium

---Dan

Thursday, March 30, 2017

New Python Course

I have been looking into Python lately, mostly with an interest in Data Science and Data Analytics. Python is a lot of fun and very powerful with access to a huge library. It has a very clean syntax which may be one of the reasons it has been gaining favor for beginning courses in computer science. So far I have not found any commercial courseware like the type available from Logical Operations and other vendors. So we are going to develop our own course! It will be based on our Introduction to Programming with Java course. We will loosely follow that topic outline using examples and practice files from Python. We will probably just use IDLE but could go with PyCharm or Anaconda or an equivalent IDE. Maybe down the road we can get into some of the neater stuff like numpy amd matplotlib, too.