Sunday, 27 September 2020

Microsoft SQL Server – first steps

Set Up

A free Developer edition of Microsoft SQL server is available from the Microsoft site.

As Microsoft change their site and methods regularly, it is probably best to follow their instructions.

It is worth getting the Microsoft SQL Management Studio at the same time as it does make a lot of the set up a lot easier.

The following will assume you have Visual Studio 2019 (Developer edition), SQL Server Express and the SQL Server Management Studio installed.

For simplicity, it is assumed that you are logged in with Windows Authentication, if not, you will need to add username and password to the database to allow access.

Also, ensure that you have set the System Administrator password in SQLServer.

My First Database

Just to be clear, this is not setting up a production database, and certainly not one ready to face a hostile world.

Start the SQL Server Management Studio (SSMS from here).

The object explorer on the left-hand side shows what it can see. 

Select the Databases node. If this is your first database, you will probably find that there is just a System Databases node inside.

You can create a database simply by right clicking on Databases and using the defaults.

This is not how databases should be created, you should have a plan written down and decide where all the files should be located. You should also consider how big the database will be initially and how the database is going to grow. Does the database need to have an upper size limit and when it grows, how much will it grow by. These can have a dramatic effect on performance (and also identify when there is a problem).

However, for this case, the defaults will be fine. Add your user as a SQLServer user via permissions – we will use Integrated Security later – this means your login is used to log in to the database.

My First Table

Efficient table design is important, you should as part of your design have considered what data you are storing, and how the data will accessed, updated and possibly deleted.

There is a process called data normalisation – for efficient storage, you should only store the information once, however it might be that in normal usage, you need certain pieces of data together, so it might be worth keeping them in the same table.

This is just about setting up a simple table, so we will initially use the SSMS to create the table.

If you click on the node for the database you created earlier it will display various nodes such as Tables, Views, Programmability etc.

Clicking on the Tables node will show you the System Tables node and possibly a File Tables node. Right click on the Tables node and select Tables.

The first data column will be an ID or Identity column. This will be an Integer (sufficient for this example). In the properties you want it not to be able to be Null, and you need to scroll down to the Identity Specification. Set that to Yes and then you can set the Is Identity field to Yes.

An Identity column is automatically filled with a value that is incremented (by default by one – there is a setting for that). It can be used to Identify the row.

The next column (Name) is a VARCHAR. This will default to 50, but 20 will be sufficient. Set that to not Null as well.

The last column is Description. This is a Text column (the contents of Text columns are not stored in the database table, there is a managed table that holds the text information. This makes the text column type efficient to store large amounts of text but at the disadvantage that the database has to be accessed at least twice, – first to get the original row that contains the reference, and second to get the text).

Viewing your data

For this, you can create a new Query Window in SSMS.

The following SQL statement will select all the contents of the table SimpleNameDescription owned by DBO in the database Testdatabase.

select * from Testdatabase.dbo.SimpleNameDescription;

Disappointingly there is no data in the database at the moment.

That is easily fixed

My First Row

Open another Query window and use the following SQL statement:

insert into [dbo].[SimpleNameDescription] ([name],[description]) values ('First one','This is the first record');

This inserts a record into the database table, setting the fields name and description to have the values listed. Note there is no mention of the ID field. The database, table and field names are enclosed in square brackets to identify they are database, table and field names.

Run the select statement from earlier (that is why a second query window was used for the insert).

Note the ID is 1. When you insert the next record, the ID will be 2. This is the great advantage of using an identity, it handles the value itself.

Using the same insert statement, add a couple of additional rows and then use the select statement to view your handiwork.

Changing a row

You have added a number of rows, but you made a mistake on one of them, how do you fix it?

SQL statements can have a Where Clause, this can be used to identify a subset of the rows in the table. Now luckily (or by good design) we have a column that identifies the rows, so we can update a specific row.

UPDATE [dbo].[SimpleNameDescription]   SET [Description] = 'new description' WHERE id = 2;

This will change the description on the row with ID of 2.

Deleting a row

Add a row using:

insert into [dbo].[SimpleNameDescription] ([name],[description]) values ('rubbish','This is rubbish');

Now as it says, this is rubbish, so you want to remove it. Use the select statement above to find the ID number

Open another query window

And add the following:

DELETE FROM [dbo].[SimpleNameDescription] WHERE id = 3;

The value 3 needs to be replaced with the ID of the row of the row you want to delete.

Run the SQL and then run the select. That row has gone. Note, that if you run the delete without the Where clause, it will delete everything and unless you have backed it up, it is gone for good. Be warned.


Programmability

Stored procedures

It is possible to build incredibly complicated systems using the SQL statements, but it has a number of security and maintenance issues. This is where Stored Procedures come in handy. These are SQL statements that can be reused.

This is a simple Stored Procedure:

SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE Test @ID int
AS
BEGIN
select * from Testdatabase.dbo.SimpleNameDescription where id=@id;
END
GO

To execute the stored procedure open another query window:

USE [Testdatabase]
GO
[dbo].Test   2;
GO

Accessing the database in C#

The following assumes you have logged in via Windows Authentication and your Windows Login has access to the database.

First you need to determine your SQL log in string.

string connectionString= "Server=localhost\\SQLExpress;database=Testdatabase;Integrated Security=true;;";

The SQL server name is your machine (localhost) and is called SQLExpress. The database is named, and it states that it will use Integrated Security. The latter hands off the heavy lifting of user names and passwords to the Windows Login. If you need to use username and passwords, replace the Integrated security with 

user Id=UserName; Password=Secret;

The overall code is:

       static void Main(string[] args)
        {
            Console.WriteLine("Connecting...");
            string connectionString= "Server=localhost\\SQLExpress;database=Testdatabase;Integrated Security=true;;";
            using (SqlConnection connection = new SqlConnection(connectionString))
            {
                string sql = " SELECT *  FROM [dbo].[SimpleNameDescription]";
                SqlCommand command = new SqlCommand(sql, connection);
                connection.Open();
                SqlDataReader reader = command.ExecuteReader();
                while (reader.Read())
                {
                    Console.WriteLine($"{reader.GetInt32(0)} {reader.GetString(1)} - {reader.GetString(2)}");
                }
                reader.Close();
            }
            Console.Write("Press any key");
            Console.ReadKey();
        }

This will display all the rows in the table SimpleNameDescription.

Now that is fine if you want to execute non selective command, but what if you want to only read for a particular ID?

 SQL Command Parameters

Though it is possible to cobble together your SQL statement, it is better to use parameters.

        static void test2(int id)
        {
            Console.WriteLine("Connecting...");
            using (SqlConnection connection = new SqlConnection(connectionString))
            {
                string sql = " SELECT *  FROM [dbo].[SimpleNameDescription] where id=@id";
                SqlCommand command = new SqlCommand(sql, connection);
                command.Parameters.AddWithValue("@ID", id);
                connection.Open();
                SqlDataReader reader = command.ExecuteReader();
                while (reader.Read())
                {
                    Console.WriteLine($"{reader.GetInt32(0)} {reader.GetString(1)} - {reader.GetString(2)}");
                }
                reader.Close();
            }
        }

Of course, as mentioned above, it is better to use stored procedures.

It is important to set the CommandType, otherwise the stored procedure will not be able to see the 

        static void test3(int id)
        {
            Console.WriteLine("Connecting...");
            using (SqlConnection connection = new SqlConnection(connectionString))
            {
                string sql = "[dbo].[Test]";
                SqlCommand command = new SqlCommand(sql, connection);
                command.CommandType = CommandType.StoredProcedure;
                command.Parameters.AddWithValue("@ID", id);
                connection.Open();
                SqlDataReader reader = command.ExecuteReader();
                while (reader.Read())
                {
                    Console.WriteLine($"{reader.GetInt32(0)} {reader.GetString(1)} - {reader.GetString(2)}");
                }
                reader.Close();
            }
        }

References

https://www.microsoft.com/en-gb/sql-server/sql-server-downloads

https://docs.microsoft.com/en-us/dotnet/api/system.data.sqlclient.sqldatareader?view=netframework-4.7.2

https://www.connectionstrings.com/sqlconnection/

https://stackoverflow.com/questions/12220865/connecting-to-local-sql-server-database-using-c-sharp


Sunday, 5 July 2020

Reading USB (Serial) data

Most microcontrollers have an option to output data over the USB link.

It is helpful during development to be able to read state information and other values that allow the developer to see what is happening.

The first thing is to identify which USB port is in use.
            foreach (string port in ports)
            {
                Console.WriteLine(port);
            }
For my set up, COM5 was the one in use.
using (var sp = new System.IO.Ports.SerialPort("COM5", 115200, System.IO.Ports.Parity.None, 8, System.IO.Ports.StopBits.One))
            {
                Console.WriteLine("Reading serial port");
                sp.Open();
                while (true)
                {
                    var readData = sp.ReadLine();
                    Console.WriteLine($"[{readData}]");
                }
            }
This will display on the console anything output from the device.





Sunday, 14 June 2020

BBC Micro:Bit Menu system (with persistent choice)

The BBC Micro:Bit is a simple microcontroller with a 5 x 5 matrix of LEDs, two buttons, 3D magnetic and acceleration sensors and a CPU temperature sensor.
The two buttons can make complicated interfaces rather difficult, however it is possible to build one.
One aadditional feature of this menu system is that your choices are persistent. If you select menu item, power off the Micro:Bit then subsequently power it back one, it will remember the choice (subject to it not being reflashed of course).

Code

This was written in Micropython using the Mu editor.
from microbit import *
import os
import utime
menuitem = 0
if 'choice.opt' in os.listdir():
    with open('choice.opt') as choice:
        menuitem = int(choice.read())
display.scroll("Menu item" + str(menuitem))
start = utime.ticks_ms()+6000
interval = 2000
while True:
    now = utime.ticks_ms()
    if start > now or now - start > interval:
        if menuitem == 0:
            temp = temperature()
            interval = 2000
            display.scroll(str(temp) + 'C', delay=100, wait=False)
        elif menuitem == 1:
            temp = temperature()
            interval = 2000
            display.scroll(str(temp + 273.15) + 'K', delay=100, wait=False)
        elif menuitem == 2:
            level = display.read_light_level()
            interval = 2000
            display.scroll(str(level) + ' light', delay=100, wait=False)
        elif menuitem == 3:
            level = compass.get_field_strength()
            interval = 6000
            display.scroll(str(level) + ' nTesla', delay=100, wait=False)
        elif menuitem == 4:
            display.scroll("Menu test")
        start = now 
    if button_a.is_pressed():
        display.scroll("Menu", delay = 100)
        sleep(50)
        while not button_a.is_pressed():
            display.set_pixel(4,menuitem,5)
            if button_b.is_pressed():
                display.set_pixel(4,menuitem,0)
                menuitem = menuitem + 1
                if menuitem > 4:
                    menuitem = 0
                with open('choice.opt','w') as choice:
                    choice.write(str(menuitem))
                display.scroll("Menu item" + str(menuitem), delay = 100)
            sleep(100)
            display.set_pixel(4,menuitem,5)
            sleep(100) 
    sleep(400)
The persistent choice is handled by this code:
menuitem = 0
if 'choice.opt' in os.listdir():
    with open('choice.opt') as choice:
        menuitem = int(choice.read())
The menu item is given a default value (0).
The file 'choice.opt' is checked if it exists in the directory list, if it is, then the value of the menu item is read from the file and assigned to the menuitem variable. When a subsequent decision is made to change the menuitem, this value is written out to the file, making it available the next time the Micro:Bit is switched on.

The main loop is entered after the start variable is set in advance of the current tick count and the display interval is set (strictly speaking the interval should be dependent on the menu choice but it only affects the first cycle).

Each loop, if the difference between the ticks now and the (loop) start ticks is greater than the interval, then the menuitem is used to choose what to do.
In this example it is used to choose which sensor is read and the results displayed.
Menu choices are:

  1. Temperature in degrees Celsius.
  2. Temperature in Kelvin
  3. Light level (based on the light falling on the LED matrix)
  4. Magnetic field strength in nanoTesla (using the compass module)
  5. A message.
The first three keep the interval at two seconds, but the magnetic field strength is a longer piece of text, so that is stretched to six seconds by setting the interval.
The Start ticks value is set to the Now value.

The next part of the code checks for the A button (left side) being pressed.
If so, it then loops until the button is pressed again.
Inside that loop, pressing the B (right hand) button increments the menuitem value, writes it to the file and shows a pixel on the right hand column indication which option is currently chosen.
Pressing button A exits the loop and recommences the outer infinite loop.

Disadvantages

This does mean that during normal operation, button A is not available. This might not be an issue but is something to bear in mind.

References


Saturday, 13 June 2020

Pimoroni Envirobit

Pimoroni Envirobit


The Pimoroni Envirobit is a set of sensors for the BBC Micro:Bit .


As you can see, it is equipped with a slot to take the Micro:Bit, so no soldering is required.

The Envirobit is fitted with the following sensors:

  • BME280 environmental sensor - which measures temperature, pressure, humidity and can calculate the altitude based on a supplied base pressure level (discuss).
  • tcs3472 RGB sensor - which measures Red Green and Blue light levels as well as “white” light levels. Also includes two illuminating (white) LEDs,
  • Sound - a small microphone allows the sound level to be measured on one of the Micro:Bit’s analogue pins

Assembly

Assembly is simple. Take the Envirobit board with the sensors facing forward, and insert the Micro:Bit with the LEDs also facing forward.
Due to the nature of the connection, you can swap the Microbits if the colour scheme does not match your needs.

Software

The main software support for the Envirobit is orientated towards the Microsoft MakeCode block based system.
There is some support for MicroPython. There is a GitHub link here: https://github.com/pimoroni/micropython-envirobit

There are three python files in the Library.
  • sound.py
  • bme280.py
  • tcs3472.py

The files can be transferred to your Micro:Bit using the Files function in Mu.

Sound

Contrary to the description on GitHub, this is not a class, just three methods.

  • sound.read() - This takes a reading of the sound level and returns a value between 0 and 440. There is an offset value in the code to set the minimum sensitivity.
  • sound.wait_for_double_clap() - listen for two high level sound events in a second, returns True if detected
  • sound.wait_for_clap() - listen for a single high sound level event in a second, returns True if detected

tcs3472

This uses a class to access the TCS3472 sensor via I2C.
To use the sensor, import the module (having transferred it to the Micro:Bit) and instantiate an instance.
import tcs3472
light_sensor = tcs3472.tcs3472() 
Methods:

  • r, g, b = light_sensor.rgb() - returns a tuple of the corrected levels of red, green and blue out of 255
  • r, g, b = light_sensor.scaled() - return a tuple of the amounts of red, green and blue on a scale of 0-1
  • level = light_sensor.light() - return a raw reading of light level on a scale of 0-65535
  • light_sensor.set_leds(0) - Turn the LEDs off
  • light_sensor.set_leds(1) - Turn the LEDs on

BME280

This uses a class to access the BME280 sensor via I2C.
The instructions on GitHub are incorrect, there is a missing () on the end of the class instantiation. Python can be very unforgiving if you make a mistake of this kind.
import bme280
bme = bme280.bme280()

The bme280 class has the following methods:

  • temp = bme.temperature() - return the temperature in degrees C
  • pressure = bme.pressure() - return the pressure in hectopascals
  • humidity = bme.humidity() - return the relative humidity in %
  • alt = bme.altitude() - return the altitude in feet, calculated against the current QNH value
  • bme.set_qnh(value) - set the QNH value for calculating altitude

QNH is the atmospheric pressure adjusted to sea level (what the pressure sensor should read at sea level).
https://en.wikipedia.org/wiki/QNH

References

https://github.com/pimoroni/micropython-envirobit
https://en.wikipedia.org/wiki/QNH


Thursday, 28 May 2020

Inkscape - SVG to PNG conversion

Inkscape is an Open Source vector graphics editing application.
The primary file type  is the Scalable Vector Graphics format. Modern browsers support the format.

However, there are still applications where bit map or raster graphic formats are required (Microsoft Azure Marketplace still wants specific resolution PNG files). Inkscape is a useful tool to convert the SVG format files to PNG.

Inkscape is in the process of rolling out version 1.0 which changes the command line parameters - so check your version.

Installation

Inkscape is available via the Raspbian repository.
After updating and upgrading the operating system, use:
sudo apt-get install inkscape

You can check the installed version.
inkscape -V
Inkscape 0.92.1 r15371

Command line usage

Inkscape has an extensive GUI, but it can also be used via command line. This is particularly useful if you want to convert an SVG file to a number of different sizes.

You can see the full list of options by using:
inkscape -?
Or a concise list with:
inkscape --usage
The command line options used:
-z No GUI
-w Width
-h Height
-e Export file name

Example
inkscape -z -w 1024 -h 1024 picture.svg -e picture.png
WARNING: unknown type: svg:foreignObject
WARNING: unknown type: i:pgf
Background RRGGBBAA: ffffff00
Area 0:0:85.039:85.039 exported to 1024 x 1024 pixels (1155.99 dpi)
Bitmap saved as: picture.png

References

https://inkscape.org/
https://en.wikipedia.org/wiki/Inkscape
https://en.wikipedia.org/wiki/Scalable_Vector_Graphics
https://inkscape.org/doc/inkscape-man.html


Monday, 25 May 2020

Setting a default screen size for a headless Raspberry PI

I have a few Raspberry Pis, and most of them are run "headless" (without keyboard or display).

Now the default screen size for a headless PI is what used to be called VGA and has a resolution of 640 x 480. This was state of the art in the 'nineties, but unfortunately the majority of software expects a larger display. This means that programs tend to fall off the available screen size, in some case losing the close button as well.

You can set the display resolution via the Raspberry Pi configuration tool or via the command line using raspi-config. The problem is that there is no guarantee that your chosen display mode will then work with your monitor.

This particuklar resolution (720p) works okay on monitors and via VNC. Other options are available.

This can be set by editing the config.txt file

sudo nano /boot/config.txt

Scroll down to this section (the highlighted text areas will probably read VGA and 1 respectively). Change the mode to 4 and the resolution description - that is for your information later).
# uncomment to force a specific HDMI mode (this will force 720p resolution)hdmi_group=1hdmi_mode=4
Use Control X, Y to save and accept the default file name to save the changes.

Reboot your machine and you should have a much more workable graphics screen on VNC.

References

Sunday, 24 May 2020

Micro:bit Beacon - part 1: the beacon

One of the features of the Micro:Bit is its Bluetooth compatible radio.

Unfortunately, it is not available in Micropython due to the size of the Bluetooth software stack.

Whatt is available is a Radio object which uses the radio hardware to communicate between Micro:bits.

There is a neat program on the documentation site called Firefly. This has groups of Micro:bits communicating with each other.

Now this project uses the "flash" technique from the Firefly project.

from microbit import *
import radio
import utime
import machine

# "Flash" effect from the Firefly program
#https://microbit-micropython.readthedocs.io/en/latest/tutorials/radio.html?highlight=Firefly#fireflies
flash = [Image().invert()*(i/9) for i in range(9, -1, -1)]

# Obtain the machine id
machineID=machine.unique_id()

# Flash the display over a half second 
# and send the machine ID every second.
while True:
        display.show(flash, delay=50, wait=False)
        radio.on()
        radio.send(str(machineID))
        sleep(500)
        radio.off()
        sleep(500)