Category: powerplatform

Record security with Power Automate

Today’s post is around record security, and how Power Automate can really be quite useful with this!

Let’s take a quick recap of how security works (which is applicable to both Dynamics 365, as well as Power Platform apps). We have the following:

  • Security roles, which are set up with specific privileges (Create/Read/Update/Delete etc) across each entity table, as well as for other system permissions
  • Users, who can have one (or more) security roles applied to them (security roles being additive in nature)
  • Teams, who can have one (or more) security roles applied to them. Users are added into the team, and inherit all permissions that the team has (much easier than applying multiple roles on a ‘per user’ basis)

That’s great for general security setup, but it does take a system admin to get it handled. Alternatively, of course, it’s possible to use AAD Security Groups which are connected to security teams within Power Platform, and users added to them will inherit the necessary permissions.

But what if we want to allow users who aren’t system administrators to allow other users access to the records? Well, it’s also possible to share a specific record with another user – doing this allows the second user to see/access the record, even if they usually wouldn’t be able to do so. This is really great, but does require a manual approach (in that each record would need to be opened, shared with the other user/s, and then closed).

I’ve been working on a project recently where we have the need to share/un-share a larger number of records, but with a different user for each record. We’ve been looking into different ways of doing this, and obviously Power Automate came into mind! We didn’t want to use code for this, for a variety of reasons.

Security and Compliance in PowerApps and Flow - Michał Guzowski Consulting

The scenario we had in mind was to have a lookup to the User record, and with populating this with a user, it would then share the record with them. This would be great, as we could bulk-update records as needed (even from an integration perspective), and hopefully all would work well.

So with that, I started to investigate what options could be available. Unfortunately, there didn’t seem to be any out of the box connectors/actions that could be used for this, which was quite disheartening.

My next move was to look at the user forums, & see if anyone had done anything similar. I was absolutely excited to come across a series of responses from Chad Althaus around this exact subject! It turns out that there’s something called ‘Unbound Actions’, which is perfect for the scenario that we’re trying to achieve.

There are two types of actions available within Power Automate:

  • Bound actions. This are actions that target a single entity table or a set of records for a single entity table
  • Unbound actions. These aren’t bound to an entity type and are called as static operations. They can be used in different ways

There are quite a lot of unbound actions available to use:

The one I’m interested in for this scenario is the GrantAccess action. More information around this can be found at https://docs.microsoft.com/en-us/dynamics365/customer-engagement/web-api/grantaccess?view=dynamics-ce-odata-9

It does require some JSON input, but when formatted correctly, it shows along the following lines:

The different parts of this works as follows:

  • Target is the actual record we’re wanting to apply the action to
  • SystemUserID is the actual system user, and we also need to specify the odatatype
  • AccessMask is what we’re wanting to do when sharing the record (as there are different options available for sharing, ie ReadOnly, Edit, ShareOnwards, etc)

Using this, we’ve therefore built out the following scenario:

  1. Field added to the record, looking up to Users
  2. Relevant users who are able to access the record can set this lookup field to be a specific user record (who doesn’t have access to this record)
  3. Power Automate flow fires on the update of the record when it’s saved (filtering on just this attribute), sharing the record with the selected user
  4. The user then gets an email to notify them that the record has been shared with them, with a URL link to it (it’s somewhat annoying that there’s no inbuild system notification when a record has been shared with you, but I guess that’s something we’re having to live with!)
  5. They can then go in & access the record as they need to

We’ve also given some thought to general record security, and have additionally implemented the following as well:

  1. If the user lookup value is changed, we obviously share the record with the new user that’s been saved to it
  2. Using a different Unbound Action (RevokeAccess), we remove the sharing of the record with the previous user (we have another field that’s being updated with the value of it, which we’re using to pass the action in, as otherwise we don’t actually know who the previous user was!)

All in all, we’re quite happy that we’ve managed to come up with this solution, which is working splendidly for us. Also, major thanks to Chad for his assistance in getting the syntax correct!

Have you ever needed to do something like this? Did you manage to implement it in some way? Drop a comment below – I’d love to hear how your experience was!

‘Ghost’ lookup value following deployment

This is something that stumped me fairly recently. It’s also something that I was trying to work out what I should use at the title for this post! Let me share what happened.

I’m working on a project that’s quite critical (COVID-19 related). This is a project that we’ve built something around Dynamics 365 as an additional wrapper, to provide specific functionality for the pandemic. It’s being rolled out (the same solution) to multiple clients, and is only using the functionality from Power Platform. No custom code at all.

Now, before going into the specifics around it, let’s take a moment to revisit what a lookup field is, and what it does. Essentially a lookup field connects two tables together (wow – that felt strange not to use the word ‘entity’!). In the front interface, it’s used for a 1:N relationship.

So for example, we can have a lookup from Account to Contact, to set the primary contact for the account. The user navigates to the field, searches for the record they’re wanting to associate, and saves it.

Underneath, there’s a relationship that’s automatically created between the two tables, showing the way that the relationship will go (ie 1:N or N:1). This is created on both sides (more on that another time around dependencies), and most people will never need to modify it

When I first started with this particular project, I got the solution, and deployed it into the Dev environment (for the project that I was on). On testing it out, I found something very interesting. We’re using the Case (Incident) table, and there are various lookup fields on it. One of these was already populated with a value. Hmm – that’s interesting, I thought. It was a new deployment, and we hadn’t set any static data up yet at all. So how could it already be populated?

How is this being set, when I’ve not entered it into the system as a record…

Furthermore, I was unable to save the Case record. When I tried to, I was getting an interesting error:

On drilling down into the error log (which admittedly is actually getting better in the details shown in it, thankfully!), it turned out to be because I didn’t have access to the referenced record (in the lookup field). It just didn’t exist.

So the lookup field value was coming in with a hard-coded GUID (record identifier). But how was this being done, especially if there weren’t any records (of that type) in the system at all?

From my experience of things, I could think of two ways in which to populate a lookup field with a hard-coded value:

  • Through a ‘real-time’ Power Automate flow, on create of the record. It’s possible to set a GUID value in the flow, and then it would be set
  • Through custom code, running on the form. Again, it’s possible to hard-code a GUID there, and then set the field

However on checking both options, none of them were happening. No Power Automate flows touching the Case record, and no custom code at all on the Case.

It was then, digging through the other parts of the solution, that I saw various Business Rules. For those unfamiliar with these, I’ll quote from the official Microsoft documentation around them:

By combining conditions and actions, you can do any of the following with business rules:

  • Set column values
  • Clear column values
  • Set column requirement levels
  • Show or hide columns
  • Enable or disable columns
  • Validate data and show error messages
  • Create business recommendations based on business intelligence.

I’ve used Business Rules (somewhat extensively) before. However on going into the one for the Case table, I found that something was happening that I wasn’t aware could happen! It’s actually possible to set a lookup field value through it:

I spy a lookup option

Even though we’ve deployed the solution from the original development environment to a different environment, this is still set. But there are no records that are available:

I had never thought that it would be possible – to set a static value (eg a number, or some text), fine. But to set referential data? Wow.

Obviously this can be quite helpful. The bit that it’s NOT helpful though is when deploying the solution to another environment (as this situation was). It doesn’t help if you re-create the record that it’s referring to with using the same record name, as it’s using the underlying GUID (which you can’t re-create). This really does take solution deployment into a whole new perspective, where you need to be careful around these sorts of things as well.

So something new that I’ve learned (I do try to learn something new each day), and specifically around an area I thought I knew quite well. It did take some time, but I’m glad that I (finally) found the root cause of it, and identified what was causing it.

Have you ever had something like this happen, where you’re searching & searching for the cause of it? Drop a line below – I’d love to hear!

Data Export Service Connection Issues

This is a slightly different post from the usually stuff that I talk about. It’s much more ‘techy/developer’ focused, but I thought it would be quite useful still for people to keep in mind.

The background to this comes from a project that I’ve been working on with some colleagues. Part of the project involves setting up an Azure SQL database, and replicating CDS data to it. Why, I hear you ask? Well, there are some downstream systems that may be heavy users of the data, and as we well know, CDS isn’t specifically build to handle a large number of queries against it. In fact, if you start hammering the CDS layer, Microsoft is likely to reach out to ask what exactly you’re trying to do!

Therefore (as most people would do), we’re putting in database layer/s within Azure to handle the volume of data requests that we’re expecting to occur.

Azure SQL Database | Microsoft Azure

So with setting up things like databases, we need to create the name for them, along with access credentials. All regular ‘run of the mill’ stuff – no surprises there. In order for adequate security, we usually use one of a handful of password generators that we keep to hand. These have many advantages to them, such as ensuring that it’s not something we (as humans) are dreaming up, that might be easier to be guessed at. I’ve used password generators over the years for many different professional & personal projects, and they really are quite good overall.

Sordum Random Password Generator Creates Random Passwords with Ease - MajorGeeks
Example of a password generation tool

Once we had the credentials & everything set up, we then logged in (using SQL Server Management Studio), and all was good. Everything that we needed was in place, and it was looking superb (from the front end, at least).

OK – on to getting the data actually loaded in. To do this, we’re using the Data Export Service (see https://docs.microsoft.com/en-us/power-platform/admin/replicate-data-microsoft-azure-sql-database for further information around this). The reason for using this is that the Data Export Service intelligently synchronises the entire database initially, and thereafter synchronises on a continuous basis as changes occur (delta changes) in the system. This is really good, and means we don’t need to build anything custom to handle it. Wonderful!

Setting up the Data Export Service takes a little bit of time. I’m not going to go into the details of how to set it up – instead there’s a wonderful walkthrough by the AMAZING Scott Durow at http://develop1.net/public/post/2016/12/09/Dynamic365-Data-Export-Service. Go take a look at it if you’re needing to find out how to do it.

So we were going through the process. Part of this is needing to copy the Azure connection string into into a script that you run. When you do this, you need to re-insert the password (as Azure doesn’t include it in the string). For our purposes (as we had generated this), we copied/pasted the password, and ran things.

However all we were getting was a red star, and the error message ‘Unable to validate profile’.

As you’d expect, this was HIGHLY frustrating. We started to dig down to see what actual error log/s were available (with hopefully more information on them), but didn’t make much progress there. We logged in through the front end again – yes, no problems there, all was working fine. Back to the Export Service & scripts, but again the error. As you can imagine, we weren’t very positive about this, and were really trying to find out what could possibly be causing this. Was it a system error? Was there something that we had forgotten to do, somewhere, during the initial setup process?

It’s at these sorts of times that self-doubt can start to creep in. Did we miss something small & minor, but that was actually really important? We went over the deployment steps again & again. Each time, we couldn’t find anything that we had missed out. It was getting absolutely exasperating!

Finally, after much trial & error, we narrowed the issue down to one source. It’s something we hadn’t really expected, but had indeed caused all of this to happen!

What happened was that the password that we had auto-generated had a semi-colon (‘;’) in it. In & of itself, that’s not an issue (usually). As we had seen, we were able to log into SSMS (the ‘front-end’) successfully, with no issues at all.

However when put into code, Azure treats the semi-colon as a special character (a command separator). It was therefore not recognising the entire password, which was causing the entire thing to fail! To resolve this was simple – we regenerated the password to ensure that it didn’t include a semi-colon character within it!

Now, this is indeed something that’s quite simple, and should be at the core of programming knowledge. Most password generators will have an option to avoid this happening, but not all password generators have this. Unfortunately we had fallen subject to this, but thankfully all was resolved in the end.

The setup then carried on successfully, and we were able (after all of the effort above) to achieve what we had set out to do initially.

Have you ever had a similar issue? Either with passwords, or where something worked through a front-end system, but not in code? Drop a comment below – I’d love to hear!

PL-200 Microsoft Power Platform Functional Consultant

Well, the last week has been quite busy, on many fronts! One of those is having a few new exams come out in Beta. I’ve already taken the PL-400 (see PL-400: Microsoft Power Platform Developer Exam for my review of it). Last Friday, the new PL-200 exam was released as well, so I scheduled it in for as soon as I could sit it.

Now the PL-200 is scheduled to be replacing the MB-200 exam at the end of this year (2020), assuming it comes out of beta by then of course. I remember sitting my MB-200, though I didn’t write up about it at the time. Compared to some of the other exams I’ve taken, it was hefty. I’ll freely admit that I didn’t pass on first go of it – it took me 3 tries to gain it! People will be required to take this as a pre-requisite for attaining the Microsoft Certified: Power Platform Functional Consultant Associate badge.

So I’ve been expecting this new PL-200 to be quite similar, but with more of a Power Platform focus. It’s still heavy on Dynamics 365, and I wasn’t expecting that part to change. The existing MB-2xx series are also staying in place (for the moment, anyhow).

According to the official description for the exam:

Candidates for this exam perform discovery, capture requirements, engage subject matter experts and stakeholders, translate requirements, and configure Power Platform solutions and apps. They create application enhancements, custom user experiences, system integrations, data conversions, custom process automation, and custom visualizations.

Candidates implement the design provided by and in collaboration with a solution architect and the standards, branding, and artifacts established by User Experience Designers. They design integrations to provide seamless integration with third party applications and services.

Candidates actively collaborate with quality assurance team members to ensure that solutions meet functional and non-functional requirements. They identify, generate, and deliver artifacts for packaging and deployment to DevOps engineers, and provide operations and maintenance training to Power Platform administrators.

The official Microsoft Learn page for the exam is at https://docs.microsoft.com/en-us/learn/certifications/exams/pl-200, and I’d highly recommend people to go check it out. I didn’t use it that much, but felt that I was on reasonable grounds with existing knowledge. It’s mostly there, but (at least in my exam) there were some sneaky extras that I was NOT really expecting. Hopefully I managed to get them (mostly) accurate!

Once again, I sat the exam through the proctored option (ie from home). The experience went without issues for once – sign in was fine, no issues with my headset during check-in, exam loaded & worked without problems at all.

So, as before, it’s not permitted to share any of the exam questions. This is in the rules/acceptance for taking the exam. I’ve therefore put an overview of the sorts of questions that came up during my exam. (Note: exams are composed from question banks, so there could be many things that weren’t included in my exam, but could be included for someone else!). I’ve tried to group things together as best as possible for the different subject areas.

  • Environments
    • Different types of environments, what each one is used for, how to set/switch them between the different types
    • How to handle security/restrict access as necessary
  • Field types. All of the available field types, what are the benefits of each, and when each type should be used
  • Data storage types. Differences between Office documents (eg Excel), CDS, SQL Server, Azure SQL. When to use each one best
  • Charts. How they’re set up, how they can be shared with other users.
  • System views. What these are, who can access them, how to set them up
  • Entity forms. The different types of forms available, how to set them up, limitations of each. When each one should be used for a given scenarios
  • Model apps. Site map. What this is, how it’s used. Implementing/customising it, the different controls available & what each one does
  • Entity editable grids
    • What these are, how they can be used, how to enable & set them up
    • Limitations that they have within the system
  • Entity/record ownership. The different types of ownerships available, benefits of each, when each should be used for a given scenario
  • Data management
    • Data importing from different sources, different methods to import data
    • What is data mapping for import, and how it’s used
  • Duplicate detection. What it is, what it does, how it works. How to implement & configure it
  • Microsoft Word templates. How they can interact with Dynamics 365, how to set them up/adjust them, what they can be used for
  • Canvas Apps
    • Expression/function types, what they are, how they’re used
    • Handling data (eg collections)
    • Offline usage & data storage
    • Controls that can be used, navigating around, loading/saving data.
  • Power Virtual Agent/Chatbots.
    • Setting them up, deploying them onto websites, deploying them into Teams
    • Configuring topics, routing, handling unknown questions
    • Bot model data, including being able to access across multiple chatbots
    • Reporting on their usage, & how customer engagements have been processed
  • Power App portals
    • Registering users, registration code process
    • Validating/confirming user accounts
    • Forms security, displaying/hiding forms & data
  • AI capabilities. AI models available. Pre-built models vs custom training, capabilities (eg text scanning), and when to use each one.
  • Omnichannel
    • What it is, when it’s used
    • How to implement, deploy & configure customers being able to be sent through to it
  • Automation
    • Workflows, Power Automate, Business Process Flows
    • What each one is, benefits/use cases for each one, when to use each for specific scenarios
  • Power Automate
    • What are triggers, & how do they work
    • What are actions, and how do they work
    • What are connectors, and how do they work
    • Prebuilt vs custom connectors, capabilities, and when to use each one
    • How to set up each type & configure them
    • Instant vs Scheduled vs Triggered
    • Security – how to enable/disable their use by users
  • Business Process Flows
    • What they are, how they’re used, limitations that they have
    • How to handle security for them
  • Business rules
    • What they are, how they’re used, how to set up/configure
    • How to use them in different parts of the system (eg forms, apps, etc)
    • Actions vs Conditions vs Recommendations
  • UI Flows (RPA)
    • What these are, how they are used
    • Requirements in order to use them
    • Desktop vs Cloud
    • Implementation, customisation, configuration & deployment
    • Limitations of them
    • Data extraction from runs
  • Security & Compliance
    • Security roles, security teams, security groups
    • What each one is, how it’s used
    • System auditing, what it is, how it’s used, how to implement & configure
    • How to access & run user audit log reports
  • PowerBI. Setting up & sharing dashboards, setting up & configuring alerts, security options/roles & how they work with data
  • Dynamics 365 integrations. What other systems can integrate directly with Dynamics 365, & any limitations that they may have

The main surprise for me was mostly around the UI flows, and the various questions I had on them. I’ve not played around with them (yet!), but they are really cool!

If you’re going to take this, I’d love to hear how your experience of it went. Drop a comment below for me to see!

PL-400: Microsoft Power Platform Developer Exam

I’ve been continuing with taking new exams as they come out. Having recently taken the MB-400 exam (see MB-400 Power Apps & Dynamics 365 Developer Exam), I was slightly surprised to see the announcement that it was going to be replaced!

Admittedly, I was also surprised (in a good way) that I passed the MB-400, not being a developer! It’s been quite amusing to tell people that I’m a certified Microsoft Dynamics Developer. It definitely puts a certain look on their faces, which always cracks me up.

Then again, the general approach seems to be to move all of the ‘traditional’ Dynamics 365 exams to the new Power Platform (PL) format. This includes obviously re-doing the exams to be more Power Platform centric, covering the different parts of the platform than just the ‘first party apps’. It’s going to be interesting to see how this landscape extends & matures over time.

The learning path came out in the summer, and is located at https://docs.microsoft.com/en-us/learn/certifications/exams/pl-400. It’s actually quite good. There’s quite a lot that overlaps with the MB-400 exam material, as well as the information that’s recently been covered by Julian Sharp & Joe Griffin.

The official description of the exam is:

Candidates for this exam design, develop, secure, and troubleshoot Power Platform solutions. Candidates implement components of a solution, including application enhancements, custom user experience, system integrations, data conversions, custom process automation, and custom visualizations.

Candidates must have strong applied knowledge of Power Platform services, including in-depth understanding of capabilities, boundaries, and constraints. Candidates should have a basic understanding of DevOps practices for Power Platform.

Candidates should have development experience that includes Power Platform services, JavaScript, JSON, TypeScript, C#, HTML, .NET, Microsoft Azure, Microsoft 365, RESTful web services, ASP.NET, and Microsoft Power BI.

So the PL-400 was announced on the Wednesday of Ignite this year (at least in my timezone). Waking up to hear of the announcement, I went right ahead to book it! Unfortunately, there seemed to be some issues with the Pearson Vue booking system. It took around 12 hours to be sorted out, & I then managed to get it booked Wednesday evening, to take it Thursday.

So, as before, it’s not permitted to share any of the exam questions. This is in the rules/acceptance for taking the exam. I’ve therefore put an overview of the sorts of questions that came up during my exam. (Note: exams are composed from question banks, so there could be many things that weren’t included in my exam, but could be included for someone else!). It’s also in beta at the moment, which means that things can obviously change.

There were a few glitches during the actual exam. One or two questions with answers that didn’t make sense (eg line 30 does X, but the code sample finished at line 18), and question numbers that seemed to jump back & forth (first time it’s happened to me). I guess that I’ve gotten used to at least ONE glitch happening somewhere, so this was par for the course.

I’ve tried to group things as best together as I feel (in my recollection), to make it easier to revise.

  • Model Apps.
    • Charts. How they work, what drives them, what they need in order to actually work, configuring them
    • Visualisation components for forms. What they are, examples of them, what each one does, when to use each one
    • Custom ribbon buttons. What these are, different tools able to be used to create/set them up, troubleshooting them
    • Entity alternate keys. What these are, when they should be used, how to set them up & configure them
    • Business Process Flows. What these are, how they can be used across different scenarios, limitations of them
    • Business Rules. What these are, how they can be used across different scenarios, limitations of them
  • Canvas apps
    • Different code types, expressions, how to use them & when to use them
    • Network connectivity, & how to handle this correctly within the app for data capture (this was an interesting one, which I’ve actually been looking at for a client project!)
    • Power Apps solution checker. How to run it, how to handle issues identified in it
  • Power Automates
    • Connectors – what these are, how to use them, security around them, querying/returning results in the correct way
    • Triggers. What is a trigger, how do they work, when to use/not use them
    • Actions. What these are, how they can be used, examples of them
    • Conditions. What these are, how to use them, types of conditions/expressions/data
    • Timeouts. How to use them, when to use them, how to configure
  • Power Virtual Agents. How to set them up, how to configure them, how to deploy them, how to connect them to other systems
  • Power App Portals. Different types, how to set them up, how to configure them, how they can work with underlying data & users
  • Solutions
    • Managed, unmanaged, differences between them, how to use each one.
    • Deploying solutions. Different methods that can be used to do it, best practise for each, when to use each one
    • Package Deployer & how to use it correctly
  • Security.
    • All of the different security types within Dynamics 365/Power Platform. Roles/Teams/Environment/Field level. How to set up, configure, use in the right way.
    • Hierarchy security
    • Wider platform security. How to use Azure Active Directory for authentication methods, what to know around this, how to set it up correctly to interact with CDS/Dynamics 365
    • What authentication methods are allowed, when/how they can be used, how to configure them
  • ‘Development type stuff’
    • API’s. The different API’s that can be used, methods that are valid with each one, the Organisation service
    • Discovery URL’s. What these are, which ones are able to be used, how they’d be used/queried
    • Plugins. How to set up, how to register, how to deploy. Steps needed for each
    • Plugin debugging/troubleshooting. Synchronous vs asynchronous
    • Component types. Actions/conditions/expressions/data operations. What these are, when each is used
    • Custom ribbon buttons. What these are, different tools able to be used to create/set them up, troubleshooting them
    • Javascript web resources. How to use these correctly, how to set them up on entities/forms/fields
    • Powerapps Component Framework (PCF). What these are, how to develop them, how to use them in the right way
  • System Design
    • Entity relationship types. What they are, what each one does, how they work, when to use them appropriately. Tools that can be used to display them for system design purposes
    • Storage considerations across different types, including CDS & Azure options
  • Azure items
    • Azure Consumption API. How to monitor, how to handle, how to change/update
    • Azure Event Grid. What it is, the different ways in which it can be used, when each source should be used
  • Dynamics 365 for Finance. Native functionality included in it

The biggest surprise that I had really when thinking back to things was the inclusion of Dynamics 365 for Finance in it. Generally the world is split into ‘front of house’ (being Dynamics 365/Power Platform), and ‘back of house’ (Dynamics 365 for Finance & Supply Chain Management). The two don’t really overlap, though they’re supposed to be coming more together over time. Being that this is going to happen, I guess it’s only natural that exam questions around each other will come up!

Overall it was quite a good exam. Some of the more ‘code-style’ questions were somewhat out of my comfort zone, and I’ll freely admit to guessing some of the answers around them! Time will tell, as they say, to see how I’ve done in it.

I hope that this is helpful for anyone who’s thinking of taking it – good luck, and please do drop a comment below to let me know how you found it!

–

Good news for Power Automate Flows!

As a starter for 10, this wasn’t actually the blog post that I was going to write today. In fact, the subject of the post wasn’t even going to be about Power Automate! However, there was some really amazing news that dropped today from Microsoft, which I just couldn’t pass up being able to talk about.

You’ve guessed it – it’s about Power Automate! Well, I suppose that the post title was somewhat of a giveaway, wasn’t it…ah well. So let’s go ahead and find out what this is all about then!

To date, we’ve been able to put Power Automate flows into a solution. Well, it wasn’t there exactly at the beginning of things, but it happened somewhere along the way. This was very convenient, as we didn’t then need to deploy each one individually to different environments. Some solutions can contain dozens & dozens of flows, and we really do love to package them all up together for ease of movement.

So that was good. But there was still a (major) ‘bugbear’ (as I like to refer to them as). This is the fact that after we deploy a Power Automate flow, we then need to go into it & (re)authenticate it. This is due to the fact that the connector/s that it uses contains what is referred to as a ‘secret’, and these can’t be moved across environments. As a result, we need to essentially recreate the ‘secret’ in the connector (ie authentication details) every time we move it. This is an annoyance (if you have one or two flows), and an absolute bloody nightmare if you have lots.

For the technical minded – every action in a flow is bound to a specific instance of a connection that it will use to “execute” that action. This is why when moving flows across environments, users are required to rebind every operation to a connection.

For example, I’ve been working with COVID-19 triage solutions. These contain lots of flows within them, connecting to multiple different sources, and doing different things. Every time we’ve performed a release (even if it’s just a simple update), we’ve needed to manually go through each flow, (re)authenticate them, and turn them on. If you forgot one, then everything can come crashing down & not work! But there’s been no other way to do it. To represent this visually, we have the following diagram

For each & every Power Automate, the connection line gets ‘broken’ when it’s deployed, and needs to be re-made.

Until now, that is. For today, Microsoft has announced the Public Preview for ‘Connection References’. Now when something is put into Preview, I usually caveat the usage of it with saying things like ‘it might go away, or not be released for a while’. But I’m going to be quietly confident about this particular piece of functionality, as I really don’t think it’s going to be pulled!

So what exactly are these? Well, in (mostly) simple terms, Connection References provide an ‘in-between’ or ‘abstraction’ layer for the connections that use them. Let’s show this visually as well

We still need to re-authenticate the Connection Reference once we deploy things. But let’s now see how we can save ourselves a massive headache, and LOTS of time:

Oooo…now this is looking better. Instead of having to update three Power Automate flows, we only have to update the SINGLE Connection Reference that’s sitting in the middle. Now multiple that by however many flows you have (eg sending emails out, etc), and start calculating how much time you’ll now be able to spend on coffee breaks, rather than doing this manually one at a time…

We can create Connection References directly from within the solution:

We then give it a name & description, choose which connector we’re going to be using, and either select an existing connection or set a new one up:

Once we’re finished, we click ‘Create’ at the bottom. Voila – we can now see it within our solution!

Note: Interestingly enough I couldn’t actually see this within the solution after I created it, even with the component selector set to show ‘All’. How I actually got them to display was changing the component selector to ‘Connection Reference’, and they then showed up. I’m thinking that this is due to it being new today/in the process of rolling out, and am expecting it to display without any issues in the near future

Let’s take a look at a Power Automate flow itself now to see how it’s referenced. When we open an item with a connector, we can now see the following:

We’re able to select the Connection Reference that we’re wanting to use. Simple, yet so powerful.

When importing a solution containing a Connection Reference, we will be prompted during the import process to set the actual connection that should be used with it:

If you don’t have any connections set up already in the environment, you’ll be able to create a new one from the dropdown.

Some things to note around this:

  • During the preview phase, Microsoft has specified that a single Connection Reference can only be used by up to 16 flows. This limitation will be removed once it goes GA
  • Existing flows will not be automatically upgraded. What you can do though is export the unmanaged solution, re-import it to the same environment, and then they will be automatically created for you. The flow/s can then be edited to update them to the correct connection reference record
  • The connection name and connection reference name are not currently synchronised. They can be different. Therefore it’s best to keep the naming conventions the same. Don’t set different names for connections and their associated connection references.

In summary – this is an awesome step forward with Power Automate functionality. I’m already tasking some of the developers on the team to re-do existing solutions to use it for ease of use. How do you think it’ll best benefit you? Drop a comment below!

Keeping belief in oneself

Although I usually post around technical matters & such, occasionally I digress into personal reflection. After all, this is my personal blog, & I feel it’s sometimes good/relevant to share certain personal things. Today’s post is along those lines, though it does relate to a technical matter.

Let’s set the scene. As many of you know (either from knowing me personally, or from reading my blog posts), I’m from the ‘model-driven app’ background. Canvas apps are really cool, but I wouldn’t say that I’m a very advanced creator of them. I’m learning the whole time about them (well, when I have a free minute here & there). There are many people in the community who are extremely more advanced than I am, and I love being able to learn from them.

I’m also considered to be in ‘Delivery’, This is the fancy word for those who run/are involved in projects, rather than selling concepts to clients. I’d run a mile if someone tried to put me in a Sales role (though I do admire the power suits that Sales have, occasionally). I’ve done a bit of Pre-Sales (where I’m helping out from a technical perspective), but haven’t been heavily involved. It’s actually something that I’m trying to work on, with being a tech evangelist. After all, if people already know/rave about the tech, how can you evangelise about it to them!!

Account Managers vs Sales People - davidmarkshaw

So last week I get a call from our Sales team. They’re really nice, and know their stuff. However they’re not ‘techies’. They had a situation – we’d been talking to a client about a potential project, and the client told us to pitch for it. Brilliant, right? Well…

The client told us that we had 4 days until the pitch deadline. Not only were we needing to pitch with the usual presentation pack (however would Sales operate without PowerPoint…?), we also had to do a live demo. Not for a completed product, but rather a Proof of Concept (PoC).

The only person available was….yes, you guess it…me. There wasn’t anyone else around with the necessary knowledge/skills to create the PoC in the time-frame needed. I’ll freely admit that I was absolutely slammed with existing projects, but wanted to be able to help out.

However, things then got ‘better’. And by ‘better’, I meant ‘interesting’. I got told who else was pitching to the client. Obviously I’m not going to mention any specific details here, but I knew who they are. More importantly, I figured that I had a very good idea of who from their side would be creating the tech, & doing the pitch.

Now as I’m not mentioning any identifiable details, I’m feeling free to say this. They’re not at my level of tech skills. They’re nowhere NEAR my level of tech skills. This is NOT because I’m better than they are. Totally the opposite – they’re SO far ahead of me with their knowledge of things, I can barely see the dust that they kick up in a race.

Knowing this, I knew that I couldn’t build a model-driven app (though it would have worked perfectly for the scenario/s we were given). I HAD to do a canvas app. But even with doing that, it wasn’t going to be anywhere near as good as what the other side would be able to put on.

The phrase ‘gibbering in fear’ does come to mind with my reaction to finding all of this out. I did feel slightly like a deer caught in the headlights. I wanted to do well, both for myself & my company, but I honestly had no idea how we could stack up.

Deer in the Headlights: By Generation Success – Generation Success
How I felt I looked like!

Thankfully, my company has an extremely open culture, and I was therefore able to talk to my manager about it. He understood where I was coming from, but encouraged me to go for it & do what I would be able to create.

My wife also encouraged me to go for it. Well, actually her words were ‘it’s not sexy when a husband says that he can’t do it, so man up and go for it!’. Ha…after that I couldn’t very well NOT do it.

So I applied myself, and with some VERY late nights (I did have other projects on, as I mentioned above), managed to get something in place. Not only did I create it, I think it looked really good. There was some really nice (canvas app) functionality, and it all came together pretty well.

Everything was in place in time (including some last minute tweaks). I even decided to spice up the demo a bit, and borrowed some dinosaurs from the kids to use for personas. We were using live camera feeds for part of the demo, and suddenly the demo was joined by ‘Rexy’, the ‘Customer Service Representative’ T-Rex! They were quite amused by it (thankfully!), and our team thought it was absolutely hilarious.

Hire A Dinosaur - Creature Events
‘Good afternoon, how may I be of assistance?’

I have no idea how the other partner pitched to the client, or what the decision will be from the client. It’s way too early for that.

What I do know is that sometimes we can lose track of ourselves. I’m not going to go into the subject of ‘Imposter Syndrome’ (check out Em D’Arcy if you want to read up about that). Rather that having others around to encourage us, even though others may be more skilled, can really make the difference.

In life, we can often face challenges. How we handle them, and how we decide to move forward, can define who we are. When dealing with technology items such as the Power Platform, where there’s constant change, it can sometimes feel very daunting, but we still need to push ahead.

Yesterday I was listening to Lisa Crosbie talking about her journey into technology (and canvas apps). As she put it – ‘there is no comfort zone here – you need to find a place to feel comfortable with this level of discomfort, and ride it to be successful’. It’s really so true. It’s not just needing to push ourselves in the traditional way, but to keep up our own confidence in our skills & abilities. With this, we can continue to drive forward, keep on learning, and continue our journey of greatness!

I’m really glad that I was able to do this, and hope that I can keep this with me. By doing so, I’ll be able to continue along my own journey.

Have you ever had a time when a challenge seemed insurmountable? How did you cope with it? Drop a comment below – I’d love to hear!

Canvas Apps, Collections & Dropdown Fields

This post is based around some recent work that I’ve been doing, which includes canvas apps. For those of you who aren’t familiar with canvas apps, imagine if PowerPoint & Excel had a baby! Though I’m expecting most people who are reading this to already know all about them 🙂

So enough with the waffle, let’s get on with things…let me paint the scenario for you.

The app is aimed to be used by a contact centre. Part of their function is to capture address information. So far this has been done absolutely manually. The issue with this is that data can be typed incorrectly, or in the wrong fields. We’re also needing to enhance the data with geographic-specific information (for reporting purposes). This information isn’t known by either the callers, or by the contact centre agents (for those who are curious, it’s the unique property reference number, which is unique to every address in the UK).

Thankfully, we’ve been given a source from the client which we can look this up against. In essence, we pass a postcode to it, and values are returned (in a JSON format). This includes the data that we’re looking for. Brilliant, so far.

When we got to thinking about things, there are several ways in which we could implement this:

  • Capture the data as we are already doing, & use Power Automate to get the relevant additional information

or

  • Automate this within the canvas app itself, and even give the customer service agents a bespoke address picker!

Deciding to go with the second option (it was a no-brainer, really), we moved ahead with this. We had the details that we needed in order to hit the address lookup API. One of the developers on the team created the Custom Connector, and got it working. We tested it out, and amazingly we got information back!

The next step was to see how we could do this within the canvas app itself. Now I’m going to admit here that although I’ve HEARD great things about Collections, I had never used them myself. In fact not only had I NOT used them before, I had NO idea how they worked! That was to change VERY quickly though…

Within a few hours, I had learned enough about collections to get how they worked, and pull data into them. It was actually really simple – I used the ClearCollect command to create a collection that was fed by the API query, which then created the data into a collection table for me to use. I was very impressed!

The code to return the postcode data. We had to do some manipulation due to the API constraints

OK – so I had my data in the collection now:

What were my next steps? Well, I was wanting to achieve the following:

  • Give the customer service agents an ‘address picker’ to use. They’d enter the customer postcode, & then be presented with a list of addresses that they could pick the correct one from
  • Automatically populate the customer address fields on the form from the selected address

Well, the first item (the ‘address picker’) was simple enough. Using a dropdown field, I pointed it at the collection data. This worked great, but the dropdown was only allowing me to select a single column from the collection to display. This meant that I could only select ONE column of data to return:

I can only select a single column!

1 column from the collection. OK, I thought – should be simple enough to handle. Let’s go and concatenate column values in the dropdown, to present the interface I’m looking for:

Now that’s more like it! Much easier for the customer service agents to use. OK – onto the next stage. Let’s go & set the fields to point to the collection, match to the value that’s selected in the dropdown, and populate. Should be simple to do, right?

Well…um, no, it’s not simple to do. In fact, it’s actually impossible to do. I was expecting to point to the dropdown selected value, & have the columns returned (from the collection). I could then select which column to use for a specific field. This, however, was not the case:

You have to love the ‘.’ (or ‘dot’) notation used in canvas app code. It shows you what values are available, and saves having to do lots of type. In this case, however, it also showed me that there was only ONE column of data to select from to display in the field. This was the ‘Result’ column.

This got me very confused. I tried going back to basics, and stripping out the concatenation in the dropdown. Wonderfully I was then presented with all of the different collection columns to use:

So let’s sum up things so far:

  • If I want to present the best option to the customer service agents (using concatenation), I can’t select different parts of the data for auto-population into fields
  • If I want to be able to auto-populate field values from the collection, I can’t use concatenation (& therefore can’t present user-friendly data to the customer service agents).

Note: Leaving aside wanting to show the house number & street, one of the main reason for wanting to concatenate was to handle buildings that had flats (aka apartments) in them. This is stored in a different column in the collection. It would therefore be difficult to show these both to the customer service agents

In essence, the behaviour of the dropdown field seemed to be that I couldn’t just change the displayed values without it ‘losing’ connection to the rest of the data. There was no ID that I could use to match on, or display what I wanted to.

This seemed to be a massive Catch-22. I tried various things, but couldn’t see a way out of this. I started to try to create a second collection, & concatenate fields from the first collection. This seemed like a good idea, though (with being totally new to it), I got lost. I tried various things; I even ended up managing to collect the entire data from the collection into a new column for EACH ROW!!

Thankfully, the community helped me out, in the forms of Peter Bryant & Clarissa Gillingham (I had posted about my issues on Twitter – the hashtag #poweraddicts is really great!).

With the help provided, I managed to work out the CORRECT syntax to use for the ‘AddColumns’ command. This now being in hand, I was successfully able to create a second collection & add concatenated field values to it:

Now for the moments of truth. Would the dropdown show this new column, & could I point the form fields to auto-populate specific columns?

Anticipation is the way to keep consumers coming back for more
Not me, but exactly how I was feeling!

The answer….was YES! It was working! I felt SO relieved. Let’s take a peek:

This was brilliant! We’re also populating other data in the background, but that doesn’t need to be visible to the customer service agents.

So in summary, I learned about collections, & how to use them. I also learned about the limitations of dropdown controls (when referencing them from other places), but came up with a way around it. Finally I achieved the result that I was aiming for. Very pleasing all round!

Have you come across something like this in an implementation? How did you manage to handle it (if you did)? Drop a comment below – I’d love to hear all about it!

Canvas Apps, Patch command, & Business Rules

Recently I’ve been doing a LOT of work with canvas apps. As I think I’ve mentioned before (at least once or twice!) my background is the traditional ‘model’ style app. As a result, it’s been quite a steep curve to skill up, but I think I’m handling it alright. I’m (slowly) getting used to the way that canvas apps work, the ability to put different controls on the screens, and reference each other.

Heck, I’m even starting to play with more advanced navigation concepts, based on some REALLY great ideas that I’ve seen (Clarissa, I can’t say how grateful I am to you for all of your assistance & guidance!).

Gradient Adventure

Amongst all of this incredible & wonderous journey, I’ve also been learning some code. Yup – you heard me correctly! I’ve always said that I’m not a developer – I respect them greatly, but I don’t develop code.

True, I’ve picked up some SQL here & there, and will freely admit that running SQL queries against the Dynamics 365 database is SO much more powerful than running an Advanced Find. Of course, it’s necessary to know the joins, conditions & such. Redgate’s SQL Helper has been amazing along the way. With moving to cloud systems, things got a little more….complicated. XrmToolBox has the SQL4CDS tool which I’ve used several times, but I was really excited by the recent announcement/release of being able to (properly) run SQL commands against the CDS database from SQL Management Studio….

Anyhow, I’m digressing. So, I’ve been needing to learn canvas app style code. It’s like Excel commands, though (slightly) different at times. Things don’t always make sense (to me, at least) – I STILL haven’t figured out why some expressions need to be in a certain order. After all, according to mathematical principles it doesn’t matter if you write A>B, or B<A. Going to still need to wrap my mind around all of this.

Simplifying Algebraic Expressions - Math 7 Quiz - Quizizz

So, one of the commands that I’m using quite frequently is the Patch command. If you’re really interested, you can check this out in detail at https://docs.microsoft.com/en-us/powerapps/maker/canvas-apps/functions/function-patch.

In short, Patch allows you to set record values from places other than a form table to the data that you’re saving. It also allows you to save field values that aren’t available on the canvas form table (due to limitations). I’ve referred to this previously at https://thecrm.ninja/canvas-app-record-set-regarding-field/. The scenario that I talk about there is just one of the things that can be done in this way. Since that post, we’ve come a long way, and are doing most things with Patch statements (due to the scenario requirements).

So that’s all well & good. However, there IS actually a reason for me writing this blog post….crazy, right? And it’s not to waffle on and on about patch statements. It’s about a very specific scenario that we hadn’t come across to date, but that came up last week.

Now, obviously you’re now VERY interested in hearing all about it, and learning for your own situations. I mean, otherwise you wouldn’t have stuck with me through this article for so long. So, let me set out what happened.

As mentioned above, we’re mostly using patch statements throughout this specific app. That’s….quite a lot of patch statements (especially as we also have IF statements governing which one is being used, as it’s not possible to use IF inside a patch statement, but I’m digress…). I’d say we’re pretty familiar with this now.

However, even with being familiar with it, we suddenly had a problem. One of the forms that we’re saving down started to NOT save down. Records weren’t being saved, which obviously is a problem!

Bear in mind here that we hadn’t touched the code for this specific action for a few weeks. Nothing had changed in our code, and nothing had changed from a platform perspective (ie Microsoft hadn’t changed any of the underlying functionality.

Going into the statement, we immediately started testing it out, and saw something interesting. We were getting an error that a required field could not be NULL:

This was quite puzzling – although in a model app we can set fields as required, and users can’t save the record until they populate it, this isn’t true in a canvas app (well, when using Patch, at least). See, it’s technically possible to use a Patch statement to create/update a record, but you don’t have to pass in required field (values). It’s a sort of workaround (& can be used in some scenarios for benefit, actually). So this happening all of a sudden was quite strange to us.

It was even stranger as we hadn’t been using the field on the form at all. The field that was being referred to was being used for a totally different process, in a different team, & not surfaced into the canvas app at all. This really was causing us to scratch our heads, and try to think (more) out of the box. It didn’t seem to be the code (we could set a value in code, but didn’t want to as it wasn’t relevant), yet we weren’t able to ignore it. Really frustrating!

With all of this in mind, I decided to go back to absolute basics after a few hours of troubleshooting. The field that seemed to be causing all of these issues was a relatively new addition, so I checked all of the details around it:

  • Was the field type correct for what it should be? Yes
  • Was it set as required on the CDS field definition? No (not that I thought this would help, but still checked)
  • Was the field on the entity form? Yes
  • Was the field set as required on the entity form? No (again, I didn’t think I’d get any joy from this)
    • Hold on….on the form designer it’s not set as Required. But when I open the form, and put some values in, suddenly it IS required.

Aha! OK – I’m now starting to see some light shining on this. I headed over to Business Rules to check out what might be there. Lo & behold, there was a business rule that set the field as required (when certain conditions were filled). An example of this would be:

Now this field hadn’t been in place when the code was developed (as mentioned above) – it had come in since. I was very curious if a Business Rule could require canvas apps to set the value, and so did some testing.

Disabling the business rule removed the error from the patch statement. Re-enabling it caused the issue again. OK – so we’ve found what’s been causing this, and could put in an adequate solution to handle it.

So in short, if you’re setting a field as being required through a Business Rule, you’re going to need to address it in any canvas app as well (that’s saving data down to the same form that it’s appearing on). Why it actually happens, when just setting it as Required on the form doesn’t, I have NO idea.

But it’s a good concept to keep in the back of your mind, I believe. Especially if there are multiple people working on developing a single entity, as otherwise you could find yourself in exactly the same scenario that we did!

Have you come across anything like this, or a different piece of strange behaviour? Comment below – I’d love to hear about i!

Power Automate & Lookup Fields

Recently I’ve been expanding my knowledge of Power Automate, and how it works. It really is a truly amazing tool, though there can be some quirks to things! There are so many connectors to use, though I haven’t really used that many of them to date.

Truthfully, most of my work in Power Automate is around CDS & Office 365. Occasionally I’ll dip into another system, but for the most part that keeps me busy enough. It’s not to say I don’t want to explore further, but finding the time can be quite difficult!

One of the great abilities that Power Automate has is to be able to update a record. With focusing on CDS entities for the moment, we would use the inbuilt action for this:

We’d run a query to get a specific record – this would give us the record ID (or GUID, depending on your preference). With this, we’d use the Update Record action & pass in the record GUID. After all, we need to know which record we’re going to update! So for example:

What we can then do is set values for the record. So we can pass in Dynamics Content, use Expressions, etc. These can be from records that are part of our Power Automate query chain, or from elsewhere.

For example, I can say that when a contact’s postcode changes (or zip code for USA), go away, look up the new city, and update it (Note: I haven’t shown the postcode lookup part below):

So this is all really brilliant. Different fields have different behaviours, of course, and we need to respect that. Otherwise the Power Automate flow won’t run, and will error. This is, of course, the digital equivalent of not trying to force a square brick into a round hole!

What we can also do is clear a field value. If for example we’re wanting to remove a value from a field, we can use the NULL expression on the field. When the Power Automate flow runs, it’ll clear whichever value the field is currently holding:

Now, one of the the field types available within CDS is the lookup field. I’m not going to go into what this is, as we should already know this!. We can, of course, set lookup fields values to populate the field, which works as expected.

However (& thanks for bearing with me so far), what happens if we want to clear a lookup field value?

Say for example that we have a task, that’s assigned out to someone. If they reject the task, we want to be able to remove them from the task record. We wouldn’t delete the task, as we still need it (& now would need to assign it to someone else). We need a way to do this.

I can hear what you’re thinking right now – mentioned above is the use of NULL, so we’d use this! Um…well, you’d think so. You can try that, but we’ve found that doesn’t always work. Additionally, that doesn’t actually seem to remove the underlying relationship that’s been put in place.

Update: Thanks to Lin Zaw Winn, who dropped me a line to let me know further information around this. The standard CDS connector (the first one that was available) allowed this to work, but the updated CDS connector (Current Environment) doesn’t allow it. Unfortunately the different connectors aren’t at parity, which is a pity!

So, there’s another way to clear lookup field values. This involves the Unrelate action that’s also available. The steps for this are as follows:

  1. Get the related record (lookup the record type, pass in the GUID for it)
  2. Use the Unrelate action to remove the connection

This will then remove the relationship, which actually results in clearing the lookup field value. In practise (for our scenario), this would look like:

Let’s take a bit of a further look at the options available here:

  • The Relationship field is the relationship between the two entities (eg here it’s Contact & Task). Thankfully you don’t need to manually type this – it’s easily selected from a dropdown list.
  • The URL field is the linked record itself

Note: It’s VERY important to have the Entity Name & URL values in the right order. I’d suggest looking up the connected record first (ie what the lookup field is pointing to), and using that as the Entity Name value. You’d then select the record where the lookup is saved on as the URL value.

What I’d usually suggest as best practise is to have a condition before this takes place. As mentioned earlier, removing the lookup would happen on a record update. This is because you wouldn’t be removing a field value if you’re creating the record!

But you’re not always going to want it removed. In the scenario that I’ve been dealing with, we’re only wanting to remove the volunteer if they’ve rejected the assigned task. So our Power Automate flow is set out like this:

  • When Task record is updated
    • Filtering on the field for ‘Task Accepted’, as we could have other things being updated on the Task record that we don’t want to trigger this particular process
  • Condition to check the ‘Task Accepted’ field value
    • When it’s something other than ‘Rejected’, cancel the flow
    • When it’s ‘Rejected’, run the Unrelate process set out above, and stop flow

You can obviously build out other functionality within it as you so desire.

So with this in mind, how do you think you could benefit from this? Drop a comment below – I’d love to hear!