Showing posts with label JSON. Show all posts
Showing posts with label JSON. Show all posts

Friday, 27 December 2019

Drinking from the Flask part 3

A Brief Interlude

So far the source has been Python and the recipient has been .Net.

Python has a number of modules to handle HTTP requests, and one of the simplest to use is the Requests module.

import requests
req = requests.get('https://<your-user-name>.pythonanywhere.com/subscribers')
print(req.text)

This will return the same data as the browser example earlier:

>>> %Run firstrequest.py
{"Subscribers":[{"subscriber": {"emailaddress": "email_0@tec.com", "subscriptionid": 0}}, {"subscriber": {"emailaddress": "email_1@tec.com", "subscriptionid": 1}}, {"subscriber": {"emailaddress": "email_2@tec.com", "subscriptionid": 2}}, {"subscriber": {"emailaddress": "email_3@tec.com", "subscriptionid": 3}}, {"subscriber": {"emailaddress": "email_4@tec.com", "subscriptionid": 4}}]}
>>>
Now once you have the data, it needs to be converted into something usable.

JSON is based on the structure of Javascript, but the Python Dictionary is easily convertible to and from JSON.

The data has a dictionary (Subscribers), within which is a list. Each item (subscriber) in the list has two name-value pairs (subscriptionid and emailaddress).

import json
import requests

req = requests.get('https://<your-user-name>.pythonanywhere.com/subscribers'

aDictionary = json.loads(req.text)
#print(y['Subscribers'])

for x in aDictionary['Subscribers']:
    print(x['subscriber']['subscriptionid'],x['subscriber']['emailaddress'])

The output:

0 email_0@tec.com
1 email_1@tec.com
2 email_2@tec.com
3 email_3@tec.com
4 email_4@tec.com

References



Saturday, 21 December 2019

Drinking from the Flask part 2

Drinking from the Flask with .Net

Background

This small project will integrate Python, Flask, JSON and .Net.

Task 1: Set up a JSON end point using Python and Flask
Task 2:  .Net application to read data from the end point
Task 3: Expand end points on the Python server for details
Task 4: Select and search for details via JSON calls.

A first sip from the Flask

This assumes you have a suitable version of Visual Studio installed, and an internet connection (plus you have a working JSON source from the previous post).

Ordering the drink

For simplicity we shall start with a Console application.

Create a new .Net Framework Console application (is it me or does Microsoft continue to add steps that add no value to their dialogues?). I called mine JSONTestProgram.

Leaving the Console program until later, I then created another project (a .Net Framework Class) in the solution called JSONClient. I renamed the class to JSONGetData. I then added an additional class file to the project called JSONSubscribers.cs.

The JSONGetData.cs file contains the following code (remember to use the URL of your Flask project).

using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using System.Net.Http;
//using System

namespace JSONClient
{
    /// <summary>
    /// This class is used to get JSON data from an endpoint
    /// </summary>
    public class JSONGetData
    {
        /// <summary>
        /// URL for getting the sunscribers information from the JSON source
        /// </summary>
        public static readonly string URLGetSubscribers = "https:// <yourusername>.pythonanywhere.com/subscribers";
        
        static HttpClient client = new HttpClient();
        /// <summary>
        /// Asynchronous function that obtains data from the URLSubscribers endpoint 
        /// and returns a dictionary of email addresses and subscription ids
        /// </summary>
        /// <returns>A dictionary with key and value of type string</returns>
        public static async Task<Dictionary<string,string>> GetSubscribers()
        {
            string content;
            Dictionary<string, string> dictSubscribers=null;
            using (HttpResponseMessage response = await client.GetAsync(URLGetSubscribers, HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false))
            {
                if (response.IsSuccessStatusCode)
                {
                    dictSubscribers = new Dictionary<string, string>();
                    content = await response.Content.ReadAsStringAsync();
                    Rootobject x = Newtonsoft.Json.JsonConvert.DeserializeObject<Rootobject>(content);
                    foreach (JSONClient.Subscriber subscriber in x.Subscribers)
                    {
                        dictSubscribers.Add(subscriber.subscriber.emailaddress, subscriber.subscriber.subscriptionid.ToString());
                    }
                }
            }  
            return dictSubscribers;
        }
    }
}
You need to add a reference to the Newtonsoft.Json package via Edit/Manage NuGet Packages.

A static instance of the HTTPClient is used in this case for simplicity (only one URL is in use).

The asynchronous task GetSubscribers will return a Dictionary of email and subscriber ids.
The GetSubscribers code first creates the dictionary to be returned.
An HTTPResponseMessage is created and initialised by an asynchronous call to the URLGetSubscribers URL.
If the response contains a status success code, the dictionary to be returned is initialised.
The content of the response is read into a string and then deserialized using the NewtonSoft component and a target class (more of which anon).
For each subscriber in the object, the email address and subscription ID is extracted and added to the dictionary.
If the response does not contain a status success code, the dictionary is returned uninitialized.

JSON data cannot be read in the same way as XML data (there is no Document Object Model equivalent), so you generally need to know the format or schema of the JSON.

This is where the JSONSubscribers.cs file comes in.
Visual Studio has added a feature which allows you to use an Edit/Paste Special command to paste text as either XML or JSON, converting the text being pasted into the required classes. When you do this, it is generally a good idea to paste it into a dedicated file - then when (if) you change the JSON file, you can just scrub and re-paste it.
This is the result in the current JSONSubscribers.cs file.

namespace JSONClient
{

    public class Rootobject
    {
        public Subscriber[] Subscribers { get; set; }
    }

    public class Subscriber
    {
        public Subscriber1 subscriber { get; set; }
    }

    public class Subscriber1
    {
        public string emailaddress { get; set; }
        public int subscriptionid { get; set; }
    }

}

The root object is used as the overall class for the conversion. The element names within the classes may not match up with the recommended naming scheme in C#, it is possible to tell it to use one name in the source and generate another in the object but that is beyond this simple example.
So, how do we call this code?

The Console program

The most complicated part of the Main method is the RunAsync().GetAwaiter().GetResult() call.
It is a lazy (and liable to deadlocks) way of calling an asynchronous call synchronously. Not really for production code.
You will need to add references to the Newtonsoft.Json package via Manage NuGet packages menu item. You also need to reference the JSONClient project.

class Program
    {
        static void Main(string[] args)
        {
            RunAsync().GetAwaiter().GetResult();
            Console.WriteLine("Press any key");
            Console.ReadKey();
        }
        static async Task RunAsync()
        {
            Dictionary<string, string> subscribers = new Dictionary<string, string>();
            subscribers = await JSONClient.JSONGetData.GetSubscribers();
            foreach (string key in subscribers.Keys)
            {
                Console.WriteLine($"{key}: {subscribers[key]}");
            }

        }
    }

The RunAysnc task calls the GetSubscribers code to obtain the Dictionary and then writes the email address and subscriber id (key - value) to the Console.
When the Console program runs, the result is as follows:
email_0@tec.com: 0
email_1@tec.com: 1
email_2@tec.com: 2
email_3@tec.com: 3
email_4@tec.com: 4
Press any key

Building a Windows Form program

This is slightly more complicated, as there are a lot of threads that can get tangled (for more details see the references).
Add a new Windows Forms project to your solution - in my case I called it JSONTestProgram.

Construct your form similar to this one (it has a label, a text box and a second button used ina later example).


The form comprises a status bar at the bottom and a tool bar at the top. There is a Docked panel in the middle with a Docked SplitContainer inside.
One of the split panels has a ListBox called lbSubscribers (which again is Docked).
The Toolbar has a number of buttons, but only the one marked Load is currently active.
The project needs to have references to the project  JSONClient and a NuGet reference to NewtonSoft.JSON.

Code wise, click on the Load Subscribers button and note the control's name.
Add:

static Dictionary<string, string> subscribers =new Dictionary<string, string>();
inside the Program class (we will be using it later).
You also need to replace the existing method generated when you clicked on the button with:

        private async void toolStripButton2_Click(object sender, EventArgs e)
        {
            subscribers = await JSONClient.JSONGetData.GetSubscribers();
            foreach (string key in subscribers.Keys)
            {
                lbSubscribers.Items.Add(key);
            }
        }

Change the tool strip button name to match the button on your form. Visual Studio will grumble about the method name, which is annoying as it created it.

That is the first part completed. Now change this to be the Start Up project for the solution and run it.
Click on the Load Subscribers button and the list of subscribers generated by your Flask project will be displayed.




References


Drinking from the Flask part 1

Drinking from the Flask

This small project will integrate Python, Flask, JSON and .Net.

Task 1: Set up a JSON end point using Python and Flask
Task 2:  .Net application to read data from the end point
Task 3: Expand end points on the Python server for details
Task 4: Select and search for details via JSON calls.

Background

PythonAnywhere is an online (Platform as a Service) integrated Development Environment and web hosting service. One of its great advantages is that there is a free version - the free version has limitations on available libraries, calls etc, but is very useful if you have a need for somewhere to develop and test Python based web content.

Flask is a lightweight web framework written in Python. It has no database abstraction layer, and is suitable when you want a simple way of generating web content from lower power hardware.

It is available on the PythonAnywhere cloud based Python environment (there is a very helpful tutorial on the site which is worth having a look at. The tutorial is so good, I will just put a link to it rather than run through it again).

Filling the Flask

Python has numerous options for developing web sites and web content delivery. Two of the most popular are Flask and Django. For this case, I will be using Flask on PythonAnywhere.
At this point I will assume that you have gone through the setting up your first database and web based Flask site tutorial from the PythonAnywhere blog: https://blog.pythonanywhere.com/121/.

One of the great advantages of Python over a number of other popular languages is that you can really write once and reuse, so first we will write some test code in your Python IDE of choice (you could use the PythonAnywhere IDE, but I am using Thonny on my trusty Raspberry Pi).

The requirement is to generate a JSON document that contains a list of email addresses and subscriber IDs.

Start off by creating a Python file called subscriptions.py.
This will contain a class to hold a subscriber called Subscriber. Because we will be using this for testing, I have added a testFill method that returns a List of subscribers (with made up names).
The Subscriber class has a method called toJSON that packages up the contents of an instance in a JSON compatible form.

class Subscriber:
    def __init__(self,email_address,subscriptionid):
        self.email_address = email_address
        self.subscriptionid = subscriptionid

    def toJSON(self):
        return {"subscriber": {'emailaddress': self.email_address,
                     'subscriptionid': self.subscriptionid}}

    def testFill():
        subscriptionsList = []
        for ix in range(0,5):
            subscriber = Subscriber('email_'+str(ix)+'@tec.com',ix)
            subscriptionsList.append(subscriber)
        return subscriptionsList

To test this, create another file called testjson.py.
This imports the Subscriber class from the file subscriptions.py and the json class from flask.
It then prints the test data out. The sort keys and indent "pretty print" the text.

from subscriptions import Subscriber
from flask import json
print( json.dumps([e.toJSON() for e in Subscriber.testFill()], sort_keys=True, indent=4))

The results look like this:
[
    {
        "subscriber": {
            "emailaddress": "email_0@tec.com",
            "subscriptionid": 0
        }
    },
    {
        "subscriber": {
            "emailaddress": "email_1@tec.com",
            "subscriptionid": 1
        }
    },
    {
        "subscriber": {
            "emailaddress": "email_2@tec.com",
            "subscriptionid": 2
        }
    },
    {
        "subscriber": {
            "emailaddress": "email_3@tec.com",
            "subscriptionid": 3
        }
    },
    {
        "subscriber": {
            "emailaddress": "email_4@tec.com",
            "subscriptionid": 4
        }
    }
]
So having successfully generated the body of the JSON document, it is now on to the web delivery.

Pouring the Flask

If you have followed the tutorial from the PythonAnywhere blog, you will have a working web site.
Now you need to add some additional pieces to deliver the JSON document.

Log into PythonAnywhere.

Select the Web tab.

Scroll down to Code and on the row Source Code click on Go to directory.

I created a new subscriptions.py file and copied the contents (you could upload it, but it was short enough that that was not an issue). I have mentioned the write once, use many times joy of Python.

Now down to the business of delivering the JSON to your eager customers.

Right click on flask_app.py and open in a new tab.

This will contain the various pieces you have added as part of the tutorial, or may be empty if you have written it yourself.

To use the Subscriber class, you need to add an import at the top of the file (you can copy and paste the one from the testjson.py file.

You will need to create a suitable endpoint.

Using Flask, you add a decorator of the form @app.route('/somewhere').
This is followed by the function that will be called when the user goes to the page somewhere on your site.

The function creates the output content from the testFill method of the Subscriber class (as previously) and then wraps it in a Subscribers class.
It then sets up the response object and returns it.

...
@app.route('/subscribers')
def testJson():
    content=json.dumps([e.toJSON() for e in Subscriber.testFill()])
    content="{\"Subscribers\":"+content+"}"
    response = app.response_class(
        response=content,
        status=200,
        mimetype='application/json'
    )
    return response
...

So, to view your work you just need to go to http://<yourusername>.pythonanywhere.com/subscribers

Your browser should display something like:
{"Subscribers":[{"subscriber": {"emailaddress": "email_0@tec.com", "subscriptionid": 0}}, {"subscriber": {"emailaddress": "email_1@tec.com", "subscriptionid": 1}}, {"subscriber": {"emailaddress": "email_2@tec.com", "subscriptionid": 2}}, {"subscriber": {"emailaddress": "email_3@tec.com", "subscriptionid": 3}}, {"subscriber": {"emailaddress": "email_4@tec.com", "subscriptionid": 4}}]}

Next: Drinking from the Flask with .Net

References