Office 365 addins as microservices on Docker containers


Office 365 Addins on Docker

This example shows how to run office 365 addins as microservices on Docker containers

https://github.com/spbreed/O365OnDocker

watch this video to see this in action

Below sections would be covered in this sample

  1. Why Docker?
  2. Prerequisites
  3. Configure Docker
  4. Docker Components
  5. Docker Commands
  6. Build NodeJS Base Image
  7. Build Office 365 NodeJS Microservice
  8. Dockerize Office365 Microservice
  9. Publish Microservice to Azure Docker Container
  10. How to run this sample

1. Why Docker?

  • Docker is an exciting new technology to build platform agnostic light weight apps/micro services/containers which can be provisioned, deployed, scaled faster than traditional VM’s.
  • Building Cloud hosted/provider hosted SharePoint/Office365 addins requires additional infrastructure to the mix.
  • Docker’s X-Plat CLI tools to provide continous integration and delivery thus making DevOps relatively simpler.
    For more details read docker docs

2. Prerequisites

  1. Windows 7 or Above
  2. Office 365 subscription
  3. Azure subscription
  4. Visual Studio Code

3. Configure Docker

  1. Install docker tool kit from docker docs
  2. Run docker quick start terminal
  3. To run this sample straightaway skip to “10. How to run this sample”.

4. Docker Components

  1. Docker Images > Blueprints of our application
  2. Docker Container > Created from docker images and are real instances of our application
  3. Docker Daemon > Building, running and distributing Docker containers
  4. Docker Client > Run on our local machine and connect to the daemon
  5. Docker Hub > A registry of docker images

5. Docker Commands

  1. $docker build
  2. $docker run
  3. $docker search
  4. $docker ps
  5. $docker images
  6. $docker rmi
  7. $docker pull
  8. $docker push
  9. $docker-machine env
  10. $docker-machine create
  11. $docker attach

6. Build NodeJS Base Image

FROM ubuntu:latest
RUN apt-get update
RUN apt-get install -y nodejs nodejs-legacy npm
RUN apt-get clean

7. Build Office 365 NodeJS Microservice

Run the sample Office 365 Add in from your local machine following this article

8. Dockerize Office365 addin

  • First install the required packages from package.json
COPY ./package.json src/
RUN cd src && npm install
  • Then copy all the source code
COPY . /src
  • Set working directory for docker daemon
WORKDIR src/
  • Set default comment on start
CMD ["npm","start"]
  • Now build the Docker Image and Run it
docker build -t o365addin-docker:0.1 .
docker run -d -p 80:3000 o365addin-docker:0.1
  • Change the AppID and Redirection URL’s in Azure and authHelper.js

  • Test the app in browser

9. Publish Microservice to Azure

1) Create SSL certs using Open SSL

openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout o365-docker.pem -out o365-docker.pem
openssl x509 -inform pem -in o365-docker.pem -outform der -out o365-docker.cer
openssl pkcs12 -export -out o365-docker.pfx -in o365-docker.pem -name "o365-docker Certificate"

2) Upload o365-docker.pem to Azure -> Settings -> Management Certificates -> Upload

3) Now Create a docker Microservice in Azure

docker-machine create -d azure --azure-subscription-id="d48ccdca-d4ab-4579-89fa-ed113033bf74" --azure-subscription-cert="o365-docker.pem" --azure-location="East US" o365ondocker

4) Connect to Azure Docker machine

eval "$(C:/Program\ Files/Docker\ Toolbox/docker-machine.exe env o365ondocker)"

5) Change the AppID and Redirection URL’s in Azure and authHelper.js

6) Build and Run the app

docker build -t o365addin-docker:0.1 .
docker run -d -p 80:3000 o365addin-docker:0.1

7) Test the app in browser

10. How to run this sample

1) Open Docker quick start terminal
2) Clone this repo

git clone https://github.com/spbreed/O365OnDocker.git

3) Update Office 365 App permissions and authhelper.js with Docker IP

docker-machine ip default

4) Build the docker image

docker build -t o365addin-docker:0.1 .

5) Run the docker container

docker run -d -p 80:3000 o365addin-docker:0.1

demystifying User Profile picture sync in #Office365


User profile pictures is one of the key factor for collaboration with in Office365. User Pictures are surfaced on almost of the service with in Office 365 such as Exchange Online, SharePoint Online, Lync (Skype) Online as well their client applications such as Outlook, Office, Skype for business etc.

To provide seamless experience to the end users its important to understand how it pictures are synchronized internally with in Office 365, Some notable issues and possible ways of automation.

Typical User Profile Sync process:

Most common scenario is of user profile picture sync process is through DirSync. When Dirsync completes the Synchronization to Azure AD, EXO gets a copy from AzureAD then SPO gets a copy from EXO. Though Outlook/Lync online is updated instantly, SPO might take up to 72 hours to show the picture.  EXO uses single image throughout the application. But SPO online uses three different images of single single user profile. For an example SPO ribbon uses a small image where as MySite/Delve uses a larger image.  So the User profile Service takes extra time to scale the images.

SPO Image variations:
200px * 200 px
_layouts/15/userphoto.aspx?size=L&username=firstname.lastname@spbreed.com

72px * 72px
_layouts/15/userphoto.aspx?size=M&username=firstname.lastname@spbreed.com

48px * 48px
_layouts/15/userphoto.aspx?size=S&username=firstname.lastname@spbreed.com

Issues:

Its not always a smooth ride from DirSync to SPO. If the process fails at any of the three stages, user’s may not get a profile pic in SPO. Here are some of the scenarios

  • AD images greater than 10KB or with wrong dimensions
  • Licensed for SPO but not EXO
  • Poor picture resolution in SPO Delve/Mysites

Resolution:

I often faced scenarios where the customers would require their pics across all the services in Office 365 in the next day. In those cases PowerShell is your friend. Below script uses Set-UserPhoto to upload high resolution pictures to EXO and ProfilePictureUploader project from Office365 Developer Patterns and Practices (PNP) to upload the Profile pictures to SharePoint Online.

Features:

Now the only thing you needed is the high resolution images on your local file share in “Firstname Lastname.JPG” format and configure the variables. This script

  • Saves the credentials securely for future use
  • Generates the UPN values from the User profile picture names. (Eg: John Doe.jpg => jdoe@domain.com)
  • Uploads the image to Exchange online
  • Scales the source image small (48px), medium (72px), Large (200 PX) images and uploads to SharePoint online
  • Archive the processed images

How to run this:

Download the Zip file and Configure the below values in configuration.xml

Configuration values:

For details please refer ProfilePictureUploader project

  1. tenantName: eg: spbreed.onmicrosoft.com
  2. pictureSourceCsv: C:\Scripts\UpdateProfilePics\Release\userlist.csv
  3. thumbs upload3Thumbs: True
  4. additionalProfileProperties:
  5. logFile path:“C:\Scripts\UpdateProfilePics\Release\log.txt”
  6. uploadDelay:500
  7. credFile:Location to store the O365 credentials
  8. psLogfile: Location to store PS logs
  9. O365AdminAccount:o365admin@spbreed.com (Global admin)
  10. localPhotosFolder:\\fs.corp.local\Public\Employee Photos\O365\ready
  11. localPhotosArchiveFolder:\\fs.corp.local\Public\Employee Photos\O365\archive
  12. domainName:spbreed
  13. exeLocation:C:\Scripts\UpdateProfilePics\Release (Location of ProfilePictureUploader Binaries)

Troubleshooting tips:

  •  O365AdminAccount should be a Office 365 global administrator
  • SetExecutionPolicy RemoteSigned should be enabled
  • Powershell modules for Azure AD and MSOL Sign in assistant should be installed.

 

AutoSPInstaller with #SP2016 on #Win2016 and #SQL2016


Since the release AutoSPInstaller by Brain Lalancette in 2010 , I am an ardent fan of this tool and used it in every other SharePoint deployment.

Not surprisingly AutoSPInstaller for SharePoint 2016 is out immediately after few days of #SP2016 Preview release.

Recently AutoSPinstaller online is release as well to generate the farm configuration file to run this tool.

For more details refer  https://autospinstaller.com/Guide

Recently I had a chance to install SP2016 on Win2016 and SQL 2016. AutoSpInstaller worked fine without any glitch and I was able to configure a single server farm in 4-5 hrs on Azure.

Here are the steps:

  1. Launch a VM using Win Server 2016 Technical preview 3 base image or install it on a Local VM2016_image_select
  2. Provision E:\ and F:\ for SQL and Logs
  3. Join the server to a domain
  4. Fully functional SharePoint install requires atleast 9 service accounts. Below PowerShell script can be used to automate Service account creation and to configure “No password expiry” policies. Update the CSV file with required values: Create-ADUsers.ps1
  5. Add replicating directory changes permission to spprofilesync account https://technet.microsoft.com/en-us/library/Hh296982.aspx
  6. Though SharePoint adds required firewall ports during setup, I used the below scripts to add required firewall ports in advance
    1. Create-FWRulesDB.ps1
    2. Create-FWRulesWFE.ps1
  7. Download and Install SQL server 2016 preview from https://www.microsoft.com/en-us/evalcenter/evaluate-sql-server-2016. Use SQLService accounts created in step 3 while configuring SQL services
  8. Pre configure app domain using http://sharepointchick.com/archive/2012/07/29/setting-up-your-app-domain-for-sharepoint-2013.aspx
  9. Download AutoSPSourcebuilder from http://autospsourcebuilder.codeplex.com/ and unzip in the same drive
  10. Download AutoSPInstaller from http://autospinstaller.codeplex.com/ and unzip in the same drive 
  11. Download #SP2016 preview from http://www.microsoft.com/en-us/download/details.aspx?id=48712 and mount it as 

    MOUNT-DISKIMAGE C:\ISO\SharePoint Server 2016 IT Preview.ISO

  12. Now navigate to AutoSPSourceBuilder  and run PS F:\AutoSPSourceBuilder> .\AutoSPSourceBuilder.ps1 -GetPrerequisites:$true  -Destination “F:\AutoSPInstaller\SP\2016”   to create SlipStreamed version of #SP2016

    AutoSourceBuilder

  13. Now prepare AutoSPInstallerInput.xml for SharePoint 2016. Its highly recommended use https://autospinstaller.com/  to generate this config today. But as of today (09/03/2016) online    tool to generate config file for SP2016 is not supported. Only notable difference wrt to AutoSPInstaller between #SP2013 and #SP2016 is the introduction MinRoles to simplify the deployment and scaling. For more details refer http://blogs.technet.com/b/wbaer/archive/2015/05/12/what-s-new-in-sharepoint-server-2016-installation-and-deployment.aspx
  14. Select <SingleServerFarm Provision=”true” /> for single server farm setup. Attached is the sample Config file for SP2016 for your reference. AutoSpInstallerInput.xml minroles
  15. Now run the .\AutoSPInstallerLaunch.bat. Installation will be a breeze.

portal

Key takeaways for #Consultants from #MSIgnite


Over the past two years Microsoft shifted its focus to deliver #CloudFirstMobileFirst Products to enterprise customers. All their product launches and announcements over past two weeks in #Build and #Ignite conferences resonates this strategy.

IMHO being a consultant working primarily on Microsoft technologies, I see this is the right time to evolve from Windows Active directory and start building against Windows Azure Active Directory as most of the advanced functionalities in core Microsoft products such as SharePoint 2016, Office 2016, Office 365, SQL server 2016 etc are targeted at WAAD as the premier identity provider to utilize Azure services such as Azure Storage, Azure service bus etc.

Here are some of the Key announcements and for more details download the videos from the links provided at end of this blog.

Product releases:
Microsoft announced release of several “2016” product lines which will hit General availability within next 12 months.
* SharePoint Server 2016
* Windows Server 2016
* SQL Server 2016
* System Center 2016
* Office 2016
* Windows 10

Azure Announcements:

#AzureStack:
Now Azure Resource Manager is the core component to create and manage all the Azure resources. Azure stack brings this Azure Resource Manager capabilities to on premise for building infrastructures based on Azure code base.
This will help the customers to build stage/test datacenters for UAT scenarios before migrating any workloads to Azure and also to build Azure ready infrastructures for future migration.

#AzureMigration
Azure site recovery evolved in to premier migration tool now supports direct migrations from VMWare to Azure or Hyper-V.

#AzureDNS
To manage DNS records using existing Azure Subscriptions.

#UserDefinedRouting
Enables a number of Virtual Appliance scenarios such as ‘NAT’ing, ‘Application firewall’, ‘Bring your own gateway’, ‘Intrusion protection systems’ etc

#ExpressRoutes
Express routes can now co-exist with Site to Site VPN and express routes is supported for #Office365

#Operations Manager Suite
To monitor any cloud, including your datacenter, Azure, AWS, Windows Server, Linux, VMware, and OpenStack.

References:

http://azure.microsoft.com/blog/2015/05/04/azure-shines-bright-at-ignite
http://azure.microsoft.com/blog/2015/05/05/new-networking-capabilities-for-a-consistent-connected-and-hybrid-cloud/ http://blogs.technet.com/b/server-cloud/archive/2015/05/04/introducing-microsoft-operations-management-suite.aspx

Office 365 Announcements:

#Office365Groups
Group managed at Office365 tenant level to ease the collaboration across every entity in Office 365 including Outlook and emails, Files and SharePoint, Messaging and Skype. All these information and interactions are surfaced in Delve using Office graph provides rich discoverability.

#Office2016
Real time co-authoring available on Office web apps are now extended to Office desktop clients.

#AzureDirectoryConnect
New and improved tool to setup ADFS, Web Proxy, Dir sync from the same interface.

#Unified API
single API endpoint and a single authentication flow for Office 365 Apps and addins.
https://graph.microsoft.com

#RemoteProvision
New community built tool to deploy Sites, Lists, Branding artifacts etc to SharePoint Online and SharePoint 2013 using CSOM and Rest interfaces

#Office365MigrationAPI
New migration API for high speed SharePoint on Prem to SharePoint Online migration utilizing Azure Storage.
Most of the ISV’s already support this.

#NetGenPortals
Integration with #SWAY to enable rich site creation experiences
Video portal integrated with Azure Media Services

#Skype
Easy Skype video meetings, skype broadcast, inetgration with hololens.

https://blogs.office.com/2015/05/04/modern-productivity-office-news-at-ignite
https://msdn.microsoft.com/en-us/office/office365/howto/office-365-unified-api-overview
https://channel9.msdn.com/blogs/OfficeDevPnP/Introduction-to-PnP-site-remote-provisioning-engine
http://en.share-gate.com/blog/what-is-the-office-365-migration-api
http://blog.cdw.com/whats-next-for-office-365-a-first-look-at-nextgen-portals

SharePoint 2016 Announcements

#Extended Boundaries
– Now SharePoint Content DB’s can be extended in TB’s.
– 100,000 site Collections Per Content DB’s
– List Items query threshold well beyond standard 5000 Items
– 10GB file uploads
– Indexing limitations pushed to 500 Million items

#Installation
– Windows Server 2012 R2, Windows 10, Windows Server 2016
– Role based install to simplify farm architectures
– User Services Role (Web Server)
– Robot services Role (App Server)
– Cache services Role (Distributed Cache)
– Special Services Role (ISV)
– Azure AD based authentication is supported to ease hybrid scenarios
– Zero downtime patching

#Cloud Search Service
– New Cloud Search service to unify SharePoint 2010, SharePoint 2013, SharePoint 2016 and SharePoint Online Indexes
– Delve Support On-Prem
– Cloud Search Service application for SharePoint 2013 editions

#UserProfileService
– Profile Sync uses Dir Sync technology instead of FIM OOB.
– External FIM can be integrated but OOB this is removed from UPS 2016

#HybridExtranet
Publish Intranet sites externally using ADFS and WAAD. No need to implement external identity providers.

#UI
– Responsive NextGen Portals
– Hybrid SP2016/0365 sites enabled via WAAD
– Improved Page load, Document download, document upload performance
– Durable links – Renaming or Moving the documents will not change the links.

SQL Server Announcements:

#Architecture changes
– Support up to 12 TB RAM
– Major changes to SQL engine to support in memory transactions on top of row based transactions to improve the performance
– Support for JSON data format

#Hybrid tables
– Stretched databases – Hybrid databases which extends portion to tables to SQL Azure
– Always Encrypted – data is encrypted by the client drivers, and unencrypted data is never seen by SQL Server

Windows Server Announcements

#Nano Server
– Light weight OS to build highly scalable Micro services and Apps
– Purpose built OS designed to run #Docker based containers and Hyper-V
– #Devops friendly OS. Supports #PowerShellDSC and #Chef

#WindowsServer2016
– Improvements to Hyper-V and Storage capabilities
– First class PowerShell DSC to ease #Devops.

I thank Vlad Catrinescu for building the PowerShell scripts to download #MSIgnite and #Build Videos and links the provided below

#PowerShell to download #Build2015 Videos: http://absolute-sharepoint.com/2015/05/the-ultimate-script-to-download-build-session-videos-and-slides.html

#PowerShell to download #MSIgnite Videos: http://absolute-sharepoint.com/2015/05/the-ultimate-script-to-download-microsoft-ignite-videos-and-slides.html

#Build2015 videos in #channel9: http://channel9.msdn.com/Events/Build/2015

#MsIgnite Videos in #Channel9: http://channel9.msdn.com/Events/Ignite/2015

TEZURE – Cost effective IOT apps using Javascript, Tessel & Azure


TEZURE

Cost effective IOT device using Javascript, Tessel & Azure

  • With the recent advancements to embedded hardware technologies, micro-controller product lines like Tessel, Espruino, RasberryPI etc where able to build compilers to breakdown Programs written in most popular programming languages in to Javascript, Python to Assembly code.

  • With Cloud providers such as Azure, AWS etc allow easy access to Queue and Storage services in a secured way, Industry experts predict that by 2020 the Internet of things (IoT) network will reach ~200 billion connected devices.

  • This blog post explains how to create a IOT device using Javascript, Tessel and Azure.

Tezure-SpyCam Demo

GitHub Site

Internet Of Things (IOT)

The Internet of Things (IoT) is the interconnection of uniquely identifiable embedded computing devices within the existing Internet infrastructure

Every Industry is getting Smarter

  • Smart Logistics
  • Smart Home
  • Smart Industries
  • Smart Healthcare
  • Smart Entertainment

With number of internet connected devices is exploding, there is a need for enabling and controlling services remotely.

Why IOT

  • Enabling services remotely
  • More targeted services
  • Data driven insights
  • Cost efficient

Tezure – SpyCam

  • Mobile controlled Surveillance Camera
    Scenario

Complexities

  • Design & Build
  • C, C++ for Sensors/Devices
  • C#, JAVA for cloud services
  • C#, Java, Obj C for Mobile devices
  • Connectivity (IP & Non IP devices)
  • Security – Identity and Authorization
  • Scalability

Keep it simple

  • Cost efficient
  • End to End solutions using pure Javascript
  • Tessel based devices/sensors
  • X platform mobile apps using Cordova/PhoneGap

  • REST based access to Azure Service bus/Azure Blobs

Hello Tessel!

Architecture
– Tessel is a microcontroller that runs JavaScript.
– It’s NodeJS-compatible and ships with Wifi built in.
– Its easier than ever make physical devices that connect to the web.
– Bridges the gap between Embedded devices and Web development

What Azure Offers

  • Azure Service bus
  • Queues, Topics, relays for messaging
  • Mobile services for push notifications and data tables
  • Azure store blobs
  • Images and Video storage
  • SAS (Shared Access Token) based authentication via REST
  • Supports Async access using NodeJS/Javascript

What PhoneGap Offers

  • PhoneGap is a free and open source framework that allows you to create X-Plat mobile apps using Web technologies
  • Mobile Apps can be created using HTML, Javascript and CSS
  • PhoneGap Build takes the pain out of compiling PhoneGap apps. Get app-store ready apps without the headache of maintaining native SDKs.

Tezure SpyCam – Architecture

Architecture

– Tessel

  • Tessel setup consists of Tessel Microcontroller and Camera module
  • Nodejs packages can be installed inside the tessel devices
  • Current set up consists of tesselPollingService() function which continously checks for new message on Azure queue
  • On new message, Camera module takes a picture and uploads to Azure blob storage
  • ‘Tesselate’ NPM module is used to load the Camera Module functions.
  • ‘Async’ NPM module is used to process the Polling cycles asynchronously
require('tesselate')(['camera-vc0706'], function (tessel, modules) {
console.log("Executed action 2 : After Camera Load : cycle no : " + count );
async.auto({
listen_messages: function (callback) {
var options = {...

Source Code

– Azure service bus

  • Azure queue receives messages from the Mobile applications and authenticated via SAS token
  • Azure Blob storage receives Images from tessel devices and authenticated via SAS
  • CORS rules can be added to Azure storage to enable X-domain requests

– PhoneGap App

  • Sends a surveillance request message with Device ID and time stamp
  • Polls the Azure Storage to retreive image with DeviceID and time stamp
  • Built with end to end javascript
    Source Code

PreRequisites

Few Gotcha’s

Provision OneDrive and set global administrator for OneDrive accounts in Office365


Recently I was working in a project to migrate all the SharePoint on-premise stuffs to Office365 including file shares. All the users had their on file shares mapped to their home drives on the PC. And the requirement is to move all the home drive data to OneDrive for Business and let the Global administrators to manage them efficiently.

Though Office 365 admin page have functionality to set Global admin for SharePoint sites, for OneDrive sites there is no OOB way to set a Global admin for all the sites. And its very tedious to set this manually for 400+ users.

Below script will help the IT pro’s to

1) Store the Office admin credentials

2) Provision OneDrive Sites for list of users

3) Assign administrator for provisioned OneDrive sites


try
{
  Get-MSOLCredentials
  Create-OneDriveSites
  Assign-Administrator
}

Download the script and change the Variables in “Input Variables” section to suit your environment


#Input Variables Begin
 #Credentials Store Location
 $credFile="C:\scripts\spbreed04cred.xml"
 #Log File location
 $Logfile = "C:\GG\OneDriveProvisionReport.log"
 #O365 Global admin
 $O365AdminAccount = "admin@spbreed04.onmicrosoft.com";
 #O365 Onedrive admin
 $OneDriveAdminAccount = "admin@spbreed04.onmicrosoft.com";
 #O365 Admin site URL
 $webUrl = "https://spbreed04-admin.sharepoint.com"
 #List of OneDrive users (seperated by CSV)
 $OneDriveUsers = "alans@spbreed04.onmicrosoft.com","admin@spbreed04.onmicrosoft.com" 
# Input Variables End

Click here the download the script.

Jumpstart AngularJS app development for O365 & E2E testing using Protractor


BDD (Behaviour Driven Development) and TDD ( Test Driven Development) has become cornerstone for modern Javascript app development but adopting these development strategies to SharePoint/Office 365 is not a trivial task.

I was intrigued to discover #Protractor, an AngularJS testing framework. With client side MVC/MVVM frameworks getting popular among the developers, AngularJS is the premier framework to develop Office365/SharePoint 2013 apps. Integrating TDD methodologies to app development model will help developers to deliver maintainable, flexible and extensible solutions and interfaces.

Below is the RevealJS presentation on how to create AngularJS based SharePoint app and execute end to end testing using #Protractor. I would like to thank Andrew Connell for introducing  RevealJS and AngularJS Apps

http://spbreed.github.io/index.html

This presentation includes training videos and Source code to build SharePoint
1) Expenses AngularApp(SharePoint hosted App)
2) ExpensesAngularApp test (NodeJS project created using VS2013)
1. Expenses AngularApp:
– This is a SharePoint hosted app developed using popular angular framework.
– Utilizes best patterns and practises for angular framework based on HotTowel framework and Learning-Path- Manager-Code-Sample from Andrew Connell.
– Uses ShareCoffee Javascript library to make REST calls to SharePoint
– Available as a nuget package for easy install
2. ExpensesAngularApp test:
– This Project runs on NodeJS and executes End to End testing on Expenses Angular App
– Utilizes Protractor framework to do E2E testing
– Jasmine is used to write test specs
Source code: https://github.com/spbreed/spangularapp