Python Selenium OTRS scraper

Python script to get CMDB entries from OTRS

Another way to do this would be to install the library functions but OTRS system management skills would be required

https://github.com/OTRS/ImportExport

So I have created a quick routine to scrape the data out.

OTRS-Web-Scraper-Clean.py

Awk script to reformat dates in text files.

Script to format a date from MM/DD/YYYY to YYYY-MM-DD



Used to reformat the date from here to make the lessons easier
https://docs.microsoft.com/en-us/sql/integration-services/lesson-1-create-a-project-and-basic-package-with-ssis?view=sql-server-2017

date-format2.sh

Azure Red Shirt 2017

My notes from the one day event.


Main presentation:

Az - command line utility Azure Mgmt -

https://docs.microsoft.com/en-us/cli/azure/install-azure-cli

L series VM for large SSD - Might be useful for I/O bound activity like Magneto debuging

Thing that allows Apps in staging to be swapped with live. And allows A B testing or say 5% of traffic to be divided. https://docs.microsoft.com/en-us/azure/app-service-web/web-sites-staged-publishing

Hockey app now owned by ms. Xamarin rich mobile app development, allows virtual testing against multiple phones. Via the test cloud

https://www.hockeyapp.net/

https://testcloud.xamarin.com/devices

Need to look at Docker

Cosmos DB introduction. https://docs.microsoft.com/en-us/azure/cosmos-db/introduction

Node.js using connotative services - Able to answer natural language questions. https://docs.microsoft.com/en-us/bot-framework/cognitive-services-bot-intelligence-overview . Node.js graph based API calls. Able to recognize faces https://developers.facebook.com/docs/graph-api/overview/

Serverless computing. Create a trigger to do something. Only the execution in billed. Example listen and respond to a github event. https://azure.microsoft.com/en-gb/services/functions/

Machine learning demo that can differentiate cat and dog images

Logic app designer in Azure - 'if this then that'

Sentiment analysis on Power bi dashboard.

Remember you can customize the AZ dashboard, remember full screen is available

Lightening talks:

1. Mindcraft hacking demo - http://www.computercraft.info/ Richard Conway using https://github.com/Microsoft/malmo https://blogs.msdn.microsoft.com/uk_faculty_connection/2017/05/23/azure-craft-a-hack-for-kids-and-parents-to-learn-ai-programming/

2. NSFW presentation (lots of swearing) about possible downs sides to cloud architecture and microseconds. Treat your services as cattle and not pets.

https://twitter.com/david_whitney

http://www.davidwhitney.co.uk/ https://en.wikipedia.org/wiki/Conway%27s_law

3. Cognitive services overview

https://www.purplefrogsystems.com/paul/2017/05/cognitive-services-with-u-sql-reference-guide/

https://www.youtube.com/watch?v=pL-c00M2CnI

https://azure.microsoft.com/en-us/services/cognitive-services/

4. Biztalk dashboard company

https://www.servicebus360.com/

5. Legacy ASP.NET discussion

Other links

https://azure.microsoft.com/en-us/azurecon/#sessions

https://twitter.com/scottgu?lang=en

https://weblogs.asp.net/scottgu

Practical Data Migration by Johny Morris

This is a great reference book for data migrations. I thoroughly recommend it for any migration project

http://www.amazon.co.uk/Practical-Data-Migration-John-Morris/dp/1902505719

The author has a somewhat abandoned blog but a useful website

http://www.bcs.org/content/conBlogPost/1852

http://dataqualitypro.com/

Perl utilities, patterns and one liners

are stored here

Counting Excel sheets and rows

To count the number of individual sheets in an Excel workbook using VBA

[Excel 2010] Go to VBA, by pressing ALT + F11. Activate the Immediate window by Pressing CTRL + G

Enter the following line and press enter.

?activeworkbook.Sheets.Count


To count number of rows in a named worksheet enter the following:

?Sheets("Sheet1").UsedRange.Rows.Count



The total number of rows across an entire Excel workbook can also be calculated using VBA. But this is how to do it using Perl.

Coding in the cloud

I have found myself using a number of different computers both on the move and at home. This is a pain. The Chrome browser has helped me bring together bookmarks and browsing history across my devices - it has worked flawlessly so far - except on a 7 inch tablet where the tabs are a little too small and the keyboard too large. But what I have wanted for a little while now is a development environment that can be accessed from any machine and anywhere.

There are of course the hosted full blooded virtual instances in Amazons and Microsoft's cloud. So far the MS Azure instances look to be the easiest to use. Remote desktop access may be blocked from certain locations. Another gotch with Azure is that if you select an Azure VM with a paid for product such as Oracle then your account will be blocked the next day and you will have to drop out of the free trail period to continue. AWS looks a little more complex to set up. Watch this space. There are web based coding environments such the Koding web based IDE using Python. See this earlier post. But these are only point solutions.

I spent some time selecting the 'best' Chromebook for me to use as a portable machine, I can not afford a MacBook Air, and small and light Windows machines are either too expensive or not worth the bother. I eventually settled on a Acer C720 I am very happy with this purchase. It is almost perfect for use on the train. The network teathering to a phone works well. So far EE have proved to provided better and faster coverage than giffgaff (O2) on the great western mainline out of Reading.
Next I'll look at the ssh client and Crouton.