Multiple ways to access your On-premise data in Microsoft 365 and Azure

If your organization is using a hybrid cloud environment, this post will shed some light to integrate on-premise resources with Microsoft 365 & Azure services. Hybrid integration platforms allows enterprises to better integrate services and applications in hybrid environments (on-premise and cloud). In this blog post, I will write about the different services & tools available with in Microsoft Cloud which allows you to connect or expose your On-premises data or application in Office 365. There are still many enterprise organizations on Hybrid mode due to various factors. It can be a challenging task to integrate your on-premises network but with right tools & services in Office 365 & Azure it can be easier. Find below the high-level overview & some references on how to

  1. Access your on-premise data in Power Platform & Azure Apps (Logic Apps, Analysis Services & Azure Data factory)
  2. Programmatically access your on-premise resources in your Azure Function app
  3. Access on-premise resources in Azure automation account
  4. Expose your on-premise Application or an existing WEB API in Office 365 cloud

Access on-premise data in Power Platform & Azure Apps (Logic Apps, Analysis Services & Azure Data factory):

The on-premises data gateway allows you to connect to your on-premises data (data that isn’t in the cloud) with several Microsoft cloud services like Power BI, Power Apps, Power Automate, Azure Analysis Services, and Azure Logic Apps. A single gateway can be used to connect multiple on premise applications with different Office 365 applications at the same time.

At the time of writing, with a gateway you can connect to the following on-premises data over these connections:

  • SharePoint
  • SQL Server
  • Oracle
  • Informix
  • Filesystem
  • DB2

To install a gateway, follow the steps outlined in MS documentation Install an on-premises data gateway. Install the gateway in standard mode because the on-premises data gateway (personal mode) is available only for Power BI.

Once the data gateway is installed & configured its ready to be used in the Power platform applications.

  1. PowerApps
  2. PowerAutomate
  3. PowerBI

The other catch the gateway is not available for the users with Power Automate/Apps use rights within Office 365 licenses as per the Licensing overview documentation for the Power Platform. Data gateways can be managed from the Power Platform Admin center.

Shane Young has recorded some excellent videos on this topic for PowerApps & PowerBI.

To use in

  1. Azure Logic Apps
  2. Azure Analysis service
  3. Azure Data Factory

create a Data Gateway resource in Azure.

High Availability data gateway setup:

You can use data gateway clusters (multiple gateway installations) using the standard mode of installation to setup a high availability environment, to avoid single points of failure and to load balance traffic across gateways in the group.

No need to worry about the security of the date since all the data which travels through the gateway is encrypted.

Data gateway architecture:

Find below the architecture diagram from Microsoft on how the gateway works

I recommend you to go through On-premises data gateway FAQ.

Integration Service Environment:

As per the definition from Microsoft an integration service environment is a fully isolated and dedicated environment for all enterprise-scale integration needs. When you create a new integration service environment, it’s injected into your Azure Virtual Network allowing you to deploy Logic Apps as a service in your VNET. The private instance uses dedicated resources such as storage and runs separately from the public global Logic Apps service. Once this logic apps instance is deployed on to your Azure VNET, you can access your On-premise data resources in the private instance of your Logic Apps using

  • HTTP action
  • ISE-labeled connector for that system
  • Custom connector

For the pricing of ISE, refer this link.

Programmatically access your on-premise resources in your Azure Function app

As you all know Azure Functions helps in building functions in the cloud using serverless architecture with the consumption-based plan. This model lets the developer focus on the functionality rather than on infrastructure provisioning and maintenance. Okay let’s not more talk about what a Function app can do but let us see on how to connect to your on-premise resources (SQL, Biztalk etc) within your function.

During the creation of a Function app in Azure, you can choose the hosting plan type to be

  • Consumption (Serverless)
  • Premium
  • App Service plan

Consumption based plan is not supported for the on-premise integration so while creating the app the hosting plan has to either premium or app service based plan & the Operating system has to to be Windows. On-premise resources can be accessed using

  1. Hybrid Connections
  2. VNet Integration

Hybrid Connections:

Hybrid Connections can be used to access application resources in private networks which can be on-premise. Once the Function app resource is created in Azure, go to Networking section of the App service to setup & configure. Go through the documentation from Microsoft for the detailed instructions to set this up.

How it works:

The Azure Hybrid Connection represents a connection between Azure App Service and TCP endpoint (host and port) of an on-premise system. On the diagram below Azure Service Bus Relay receives two encrypted outbound connections. One from the side of Azure App Service (Web App in our case) and another from the Hybrid Connection Manager (HCM). HCM is a program that must be installed on your on-premise system. It takes care of the integrations between the on-premise service (SQL in this case) with Azure Service Bus Relay.

Once the setup is done, you can create a connection string in Appsettings.json file or from Azure function app interface of your function app. After this you can access the data in your function app code.

I’ve found a couple of interesting blogs about this setup.

VNet Integration:

In the Networking features of the App service, you can add an existing VNET. An Azure Virtual Network (VNet) is a representation of your own network (private) in the cloud. It is a logical isolation of the Azure cloud dedicated to your subscription.

In Azure Vnet you can connect an on-premise network to a Microsoft VNet, this has been documented from Microsoft here. Once there is integration between your Azure Vnet & on-premise network and the VNet is setup on your function app you are set to access on-premise resources in your function app.

Access on-premise resources in Azure automation account:

Azure Automation is a service in Azure that allows you to automate your Azure management tasks and to orchestrate actions across external systems from right within Azure. Hybrid runbook worker feature allows you to access on-premise resources easily. The following diagram from Microsoft explains on how this feature works

I’ve written a blogpost recently about this feature for automating on-premise active directory.

Expose your on-premise Application or an existing WEB API in Office 365 cloud:

Azure Active Directory’s Application Proxy provides secure remote access to on-premises web applications (SharePoint, intranet website etc). Besides secure remote access, you have the option of configuring single sign-on. It allows the users to access on-premise applications the same way they access M365 applications like SharePoint Online, PowerApp, Outlook etc. To use Azure AD Application Proxy, you must have an Azure AD Premium P1 or P2 license.

How it works:

The following diagram from Microsoft documentation shows how Azure AD and Application Proxy works

Find below documentations on how to

  1. Add an on-premises application for remote access through Application Proxy in Azure Active Directory
  2. Secure access to on-premises APIs with Azure AD Application Proxy
  3. Use Azure AD Application Proxy to publish on-premises apps for remote users
  4. Deploy Azure AD Application Proxy for secure access to internal applications in an Azure AD Domain Services managed domain

Once the connector service is installed from your Azure AD application proxy, you can add an on-premise app as shown below

The above step will register an application with App registrations.

Summary: I’ve given some overview about the different services & tools to connect & integrate on-premise resources with Microsoft cloud. Hope you like this post & find it useful. Let me know any feedback or comments on the comment section below

Restricting Power Apps users from viewing or updating the SharePoint List content from SharePoint GUI

There are many developers & power users using SharePoint lists as a data source to store data submitted from a Power App. There should be various reasons for this, please find mine below

  • Enterprise users with license (E5, E3, E1) has access to Power App & SharePoint as a service
  • SharePoint is a standard connector & not a premium one which means the user would be able to access SharePoint with the license they have assigned. Its not required for the users to have an additional license (App based license or user based license) to access the premium connectors like Azure SQL etc
  • There are many triggers & actions available for SharePoint connector

Let’s say on the Power App you have developed or developing has many custom logics on different screens. For e.g data being stored on different lists on click of a button in a Power App screen/form. If the user has to store data to SharePoint from Power App, user should have at the least edit/contribute access which means the user to be added on the out of the box site members group.

If the power app user gets to know the SharePoint site url which is connected as a data source in the Power App, the user would be able to access all the lists from the site by going to the Site contents page or by directly going to the list view url. This might create an issue because the user might edit, create or delete an item from the SharePoint interface rather from the Power Apps interface. As an app owner, you would like to avoid this. There is a way to avoid this by creating a custom permission level for the app users in SharePoint. Please find the steps below

  1. Go to the Site Permissions
  2. Click Permission Levels on the Ribbon
  3. Click an existing permission level “Contribute” from the list.
  4. You will be navigated to Edit Permission level for “Contribute”
  5. Scroll the page to the bottom & click on the button “Copy Permission Level”
  6. Name the permission level as “Power Apps – Custom Permission” & give some meaningful description
  7. Remove/Uncheck the permission “View Application Pages” under the category “List Permissions”
  8. Scroll down and click “Create” button

The custom permission level is ready, let’s assign this to a new SharePoint group. Find steps below to create a new group & assign the custom permission level to the group

  1. Go to the Site Permissions
  2. Click “Create Group” on the ribbon
  3. Name it as “Power App Users”
  4. Change the Owner field to the Site Owners group – Optional Step
  5. Scroll down & select the Permission level created above
  6. Click Create

The group is ready, add the power app users to this group. This access will not allow the members of this group to access the Site contents (Application Page), view forms or views (Allitems.aspx) but they would be able to create/update from Power Apps. Technically the user would still be able to access/modify the list content programmatically using CSOM/JSOM etc, but the GUI in SharePoint is blocked. They will receive an access denied message while trying to access the list view/form or site contents page.

Summary: You can play around with the different types of permissions while creating the permission level for Power Apps users, grant only the required permissions. A site owner will have access to create or update permission level. Microsoft has very detailed documentation on permission levels. In a SharePoint site, the permissions are inherited from the Site>Lists>Item. Based on the need, the permissions can be broken at any level. Hope you find this post useful. Let me know if there is any comments or feedback by posting a comment below.

Automate the backup of your Flow & Power Apps canvas application

Microsoft Power Automate & Power Apps have turned quite important & popular service now a days within Office 365, there are many users (developers & IT pro users) who are creating business applications leveraging these services very quickly but if an app or a flow is deleted by mistake then its not an easy job to spin it back up with out involving Microsoft support. In this blog post, am going to give you some ideas to back up a Flow & Power Apps automatically with the help of connectors available in Power automate. Once it is backed up, I will show you steps to restore from the back up.

Once I’ve deleted a flow accidentally and I was not able to restore it back, I have recreated the complete flow with all the steps which took some time. I will show you steps to back up & restore your Flow & Power Apps.

  • Backup & Restore your Flow
  • Backup & Restore your Power Apps

Backup & Restore your Flow:

A flow could be backed up by copying the flow definition & connection references with the help of an action “Get Flow” under the flow connector “Flow Management”. With the flow definition & connector references we would be able to restore it back whenever needed. Thanks to John Liu for the tips & guidance on showing us a way to copy the flow definition. Before showing you steps to get the flow definition & connection references, I will show you some examples on the type of information it stores on these fields.

Flow Definition:

It has details about the triggers & actions used with in a flow in a JSON format.

Flow Definition

Triggers: Information about the trigger used in the flow (Instant, Automatic, Scheduled)

Actions: Information about all actions used in the flow

Connection References:

It has details about all connections used against each action

Connection References

Add the action “Get Flow” under connector “Flow Management”, which has the fields flow definition & connection references. This action would be able to get only the flows to which you have access to. If you don’t see the flow under the drop down, enter the Flow ID by selecting custom value.

Now let’s store the Flow definition & connection references to a file in some location so that we would be able to retrieve the files for restoration. On this example I will use “One Drive for business connector” action “Create File” to store the content in two files but you can also get it stored on SharePoint Document Library, Azure Blob, GitHub etc. The extension of these files can also be .txt but I’ve used .json.

One Drive for Business connector

Restore Flow:

Flow action “Create Flow” helps us to restore flow from the flow definition & the connection references stored on One drive. Only the file content should be passed for the Flow Definition, connectionReferences and not the actual file.

Backup & Restore your Power App:

A Power App could be backed up by creating a MSAPP file for the PowerApp to be backed up. The file with extension .msapp is just a ZIP file which has all the definitions of the app. There are open source tools (Review Tool, Theme editor, App merger, Phone to tablet converter) available in github which helps us to peek at the definitions of the .msapp file.

PowerApps Review Tool

PowerApps for App Makers connector has an action called “Get App” which helps us to get all the information related to the specified app. This action has a field called “readonlyValue”. It points to a link that has the PowerApps msapp file. With the help of the premium connector HTTP, we will make a GET request to the readonlyValue link (Blob storage path). This step allows you to retrieve the contents of the msapp file which could then be stored as a backup file in Onedrive, SharePoint, Github etc. Lets now look at the contents of the msapp file, change the extension of the .msapp file to .zip file and open it.

AssetsAll the media files (Images, Videos, Audio)
ControlsInformation about all the controls (First Party, Custom components etc) used in the different screens of the app. Data is in JSON format.
ReferencesInformation about Datasources, Templates (Label, Gallery, Textbox etc), Resources (Images etc), Themes used in the app etc
ResourcesPublish information, user locale etc
Properties.jsonInformation of the app like Author, PowerApp name, no of screens etc
Properties.json file

Add the action “Get App”, enter the App ID of the Power App

Get App action

This action retrieves all the information of the app including the “readonlyvalue” as explained above. Using this result we will make a GET request using the HTTP connector to retrieve the app data in this Flow.

HTTP (Premium connector)

Add the action “Create file” in OneDrive for Business connector and set the body from HTTP connector. This allows you to save your app to OneDrive for Business. As said earlier you can instead store it to other storage like Azure Blob Storage, SharePoint document library, GitHub etc.

Restore Power App:

The msapp file is with us now, lets now restore the PowerApp. Create a Blank app from the Power Apps studio then open the.MSApp file via the File > Open > Browse menu option. This enables you to select the .MSApp file from your local file system then save & publish the App. The app is now back. Let me know on the comment section if there is a automated way to restore the PowerApp from the MSApp file.

Summary: On this blog post, we have seen ways to backup & restore your app & flow using a Flow. If you are an Power App/Power Automate environment admin (Premium license), you could backup all the apps & flow available in the environment using the Apps for Admin connector. There are Power Shell cmdlets available for Admins & creators which could also be used. If you have a more complex business application including PowerApps Canvas, multiple flows, custom connector, Model driven apps, entity etc, solutions is recommended. Exporting a solution as a package is very easy by click of a button, restoring is also quite an easy task. Hope you find this post useful, let me know your feedback on the comments section.

Managing users for a Power App with SharePoint as a data source

In this blog post, I am going to show you how to manage users for a Power App which has SharePoint as a data source. Let’s take an example, you’ve built a Power App application which has its data stored in SharePoint. After the application is developed, you want to share the app with some users. To do so you’ll have to give the user access to

  • PowerApps
  • Access to the Data source (SharePoint in this case), it could be Read/Write based on the use case

Only after the user has been granted access, they would be able to use the application. To setup the process for managing users we are going to use an action “Edit App Role Assignment” under the connector “PowerApps for App Makers” in the flow. Find the steps to executed to make this happen

  1. Create an Interface/screen for the Admins in PowerApps (custom role) to maintain (Add/Remove) the users of the application
  2. Flow for granting access to the users
    1. Action to grant access to the PowerApps
    1. Action to add the user to the SharePoint site

Step 1: Create an Interface for the Admins in PowerApps to manage the users for the application

Create a list (User Roles) on the SharePoint site which acts as a data source for application with the below schema to maintain the users

ColumnType
UserNamePerson or Group
RoleChoice (value: Admin, User)
RoleStatusSingle line of text (Default value: Add)
List Schema for UserRoles

Add a screen on the Power App as shown on the image below. I’ve given some information on the image the type of controls I’ve added on the screen.  This screen will be shown only to the Admin role, the navigation to this screen could be based on click of a button placed somewhere on the screen. The visibility of the button to navigate to the admin interface could be set based on the data (Role) from the list User Roles list. In this example, I will have two roles an Admin and User. Find the below screen in PowerApp built for managing the users

PowerApp Admin Interface screen

Once the user name and the role has been entered on the section 1, click add which adds the user information to the list User Roles. Once a user is added, the flow will be triggered which has actions to grant access for a Power App.

Step 2: Flow for granting access to the users

I will be using a flow to grant access to PowerApp & SharePoint site. Let’s build the automated flow with trigger “When an item is created or modified” connected to the list User Roles

Automated Flow Trigger

Add the action “Get User profile”, the input should be the email address of the user who must be given access to PowerApp. This step is required to get the user guid, to be given on a later step for granting access to PowerApp action

Get User profile

Now Add the action “Edit App Role Assignment” under the connector “PowerApps for App Makers”, this action is in preview mode by the time I am writing this post. To get more details on the different actions and its parameters with this connector go through this link. Make sure the connection to this action has access (Owner/Co-Owner) to share an App to a user.  You would be able to get the GUID (App ID) of the Power App by going to the details section of an app.

Edit App role assignment flow action

Now let’s add an action to grant the user access to SharePoint site. I would be using a flow action “Send an HTTP request to SharePoint” to call a Rest API (POST) to get the user added to the SP site groups (Members/Viewers/Visitors). The connection for this action should have access (Site Owner) to share the site. Find the rest api details to add the user to a group

URI: _api/web/sitegroups(groupId)/users

Method: POST

Request Body: {‘LoginName’:’i:0#.f|membership|Emailaddressoftheuser’}

To get the SharePoint group id, navigate to the url https://domainname.com/sites/sitename/_layouts/15/user.aspx and click the group name and copy the URL which will have the groupid at the end.

https://domainname/sites/sitename/_layouts/15/people.aspx?MembershipGroupId=3

Find the action with the configurations

SP HTTP request action

Execute the flow by adding the user to the list from the PowerApps interface, the flow gets triggered which will in turn grant the user access to PowerApp & SharePoint. Tada!!!

To revoke the access to a user, if you scroll back to the admin interface image for managing the user there is a delete button. Assume that the delete button click will update the column status on the User Roles list to “Revoke” for a user which would then trigger the same flow. On the same flow add a Switch which would then based on the value in the RoleStatus (Add/Revoke) column, add steps to revoke the access. This could be achieved in multiple ways. For e.g After the delete button is clicked the item on the UserRoles list can be deleted which will in turn call a flow created using the trigger “When an Item is deleted” with actions to revoke access

Action to Revoke Power App access, its the same action we used for adding user to an app but with some different parameters as shown below

Edit Role assignment – Remove user

The rest api details for revoking the user access from a SharePoint group

URI: _api/web/sitegroups(groupId)/users/removeByLoginName

Method: POST

Request Body: {‘loginName’:’i:0#.f|membership|Emailaddressoftheuser’}

The action to grant access to Power App can also be called from a Power App by creating a connection to the data source “PowerAppsforAppMakers” as below

Editapproleassignment from powerapp

Summary:This approach would help the super users of the app to manage user permissions for the Power App within Power Apps interface. On this post we have seen adding the user permission to SharePoint data source. Based on needs there should be ways to add users to different data sources if there’s an api endpoint or with the help of standard connectors available in Power Platform. Let me know any feedback or comments on the comment section below