First, the background. I’ll explain the problem and the goal. Later I list all of the steps to build out the CI/CD process.

Background

I recently started using the Ionic framework to develop cross-platform mobile apps. I like the idea of using HTML with Angular and TypeScript because we are already doing that for our web apps. (I’m not going into the pros and cons of a hybrid mobile app or any of that stuff, that is not what this blog post is about.) Anyway, along with Ionic, I evaluated Ionic Pro, a paid service that provides some pretty cool features for teams building Ionic Apps. For my non-mobile apps (Web Apps, etc.), I have testing environments named SysTest, QA, UAT, and of course, Production. Those multiple environments can be tricky to manage for mobile app development. Ionic Pro can help here and in particular, I like Ionic’s features: View App, Deploy, and Package. These tools make it pretty easy for us to test our mobile apps through all of those phases and then deploy it for production. Read up on the View App – it is pretty cool. However, I think their implementations, while a great start and very useful, fall pretty short of what I really need to create mobile apps in an enterprise development environment. But I knew I could get all of my tools to work together and it took some work but I’ve got a good model now that I wanted to share with the world.

There are a lot of possible ways to do this. I’m using TFS and Octopus Deploy and I am assuming that you, the reader, are familiar with those products already. If you use a different CI server or deployment system, I’m sure you can take the concepts from here and apply them to your environment.

My Current CI Pipeline for other (non-Ionic) apps

First, an overview of my DevOps pipeline for my other apps. I think this is a fairly straight forward/typical model.

  1. Developer makes a change locally.
  2. Developer builds locally to test application, including debugging, etc..
  3. Developer commits changes to source code repository (Git in TFS).
  4. Commit triggers a CI build:
    1. Gets code from source control
    2. Loads dependencies such as NuGet, NPM, etc..
    3. Compile
    4. Test
    5. Packages the application
    6. Publishes the Package to Octopus Deploy and create a release
  5. When triggered, Octopus Deploy updates configuration settings (variables, connection strings, etc.) for the specific environment and deploys the packaged application.
  6. Users can easily trigger step 5 for any of the environments (SysTest, QA, UAT, Prod, etc.). If you are new to this, remember that when Octopus deploys, it uses the same package for each environment. So we never go back to step 4.

Another note on the current process: I like to be able to trace my changes from checkin to build to deployment. One way I do that is to use the same value for the TFS Build Number, the Octopus Package version, the Octopus Release Number, and assembly version number.

I’ve simplified the process a bit for purposes of this post but here are some key takeaways:

  • After a developer commits the code, they don’t touch it again.
  • The code is compiled once on the server (not on a desktop machine by a developer which could introduce problems)
  • The same compiled application is deployed to each environment
  • The only changes are the configuration values managed by Octopus.
  • It works GREAT!

My goal was to get as close to this set up as possible. I think I did.

Unfortunately, Ionic Pro’s feature isn’t rich enough

Doesn’t Ionic Pro already allow you to move code through different environments? Sort of – but it is not good enough. What does Ionic Pro offer? With Ionic Pro, users can push the code to Ionic by committing and pushing changes to a Git repository hosted by Ionic. This is not meant to replace your normal Git repo. I think this is a pretty creative and easy way to push code to Ionic. Ionic will then build the project for me. Ionic supports multiple channels (sort of like environments). After code is committed to Ionic and then built by Ionic, the build can be automatically or manually deployed to a channel. With the Ionic View app on a mobile device, you can easily test versions from each channel. You can create any channels you like (we used SysTest, QA, UAT, and Prod) and Ionic Pro suggests testing the code in each area and then promoting the build to the next environment. The problem with this model is that if I “promote” the build from SysTest channel to QA channel, there is no way to change configuration variables. So while it seems cool, this is pretty much useless to me. Most likely any I am running or testing in SysTest, QA, UAT, or Prod will have different users, permissions, data, passwords, database connections, logging systems, etc. for each environment. After a bit more research I figured out that Ionic supports Git branches. Ok, now this is getting interesting and my brain starts trying to figure out how to make this work. In Ionic, each commit triggers a build. And the build can be deployed to a channel. So the next obvious idea (and one I think Ionic promotes) is that if developers need different settings for each environment, they should change the variables on their dev box and then push the changes to a new branch. The flow would look like this:

The Prescribed Ionic Pro Design(no good)

  • Dev makes a change.
  • Developer builds locally to test application, including debugging, etc..
  • Developer checks code into Git repo (not Ionic, I mean TFS, GitHub, whatever)
  • Dev checks in change with SysTest variables to the SysTest branch in Ionic.
  • Ionic builds and deploys to the SysTest channel.
  • After testing in SysTest, the developer would update the variables for QA locally and then commit the changes again, this time to the QA branch.
  • The developer must make sure to check in the correct code without any updates since the previous SysTest deployment.
  • (I assume the developer does not check these changes into the normal source control repo.)
  • Ionic builds and deploys to the QA channel.
  • Etc., etc.

My Design

While the previous model could work for some people, it is error prone and seriously flawed. There is too much manual intervention and too many chances for human error to occur. Each time a developer checks in the changed configuration values there is a risk that she or he will introduce a bug. So I came up with this better, more automated, harder to set up, but certainly safer and less error prone plan:

  • Developer makes a change locally.
  • Developer builds locally to test application, including debugging, etc..
  • Developer commits change to TFS Git repository.
  • Commit triggers a CI build:
    • Gets code from source
    • Loads dependencies such as NuGet, NPM, etc. This is done later by Ionic Pro
    • Compiles This is done later by Ionic Pro
    • Tests
    • Packages the application
    • Publishes the Package to Octopus Deploy and create a release
    • (note this isn’t really a “build” but it uses TFS’s build system.
  • When triggered, Octopus Deploy updates configuration (variables, connection strings, etc.) for the specific environment and “deploys” it. However, in this case, “deploy” means to take the code and commit it to the specific Ionic branch as needed.
  • Ionic then builds the branch and deploys the branch to the appropriate channel.
  • Octopus and Ionic repeat the process as needed for each: SysTest, QA, UAT, Prod (etc.)

Here is my same list of takeaways, with notes for the new design:

 

  • After a developer commits the code, they don’t touch it again. Yup
  • The code is compiled once on the server. Not exactly. It is compiled on the server but not once. It gets compiled for each environment
  • The same compiled application is deployed to each environment. Nope
  • The only changes are the configuration values managed by Octopus. Yup
  • It works GREAT! Yup!

My design also has some big flaws but I think the compromises are pretty good. For example, I’d like the code to be built once instead of built for each environment. But due to the nature of Ionic apps and Ionic Pro, we are stuck. The good part is that while we are doing multiple builds, they are all done by the same user/environment (Ionic Pro), from the same source code (it is packaged once), and it is all automated with no human interaction.Another weird thing is that my TFS build isn’t really building it and my Octopus Deploy isn’t really deploying a finished product (it just pushes code to a Git remote). But on the other hand, if you were to compare this process to my other projects, there is a lot of consistency. And I can still use the Octopus dashboard to see the status of my app in different environments.

How to set this up

This doesn’t sound too complicated but parts were a pain to figure out. Here are the steps to set up:

Git

Of course, we need a Git repo. Do what you want, I put mine in our on premises installation of TFS. Pretty straightforward, I won’t provide instructions.

Source Code

Add the source code for your Ionic project to your Git repo. Again, I am not providing instructions on how to use git.

Link to Ionic Pro

Link your Ionic project to Ionic Pro (I assume you have an account already) with a command like: ionic Link –pro-id YOUR_ID

Build

Next, we need our Build. Again, it doesn’t really build anything but it is the heart of the Continuous Integration pipeline. I use TFS but you can use what you want to accomplish the same thing. The build has 6 steps:

clip_image001[4]

Here are the steps. Of course, you can choose to do this differently, especially the 4 Octopus Deploy related steps.

a. Get sources

clip_image002

b. Update Octopus Package Version

I have a variable that I use for setting the Octopus Package number and release number. It is based on my TFS Build Number but needs to be set with PowerShell.

c. Package

Yes, package is the next step. I don’t actually build anything. Ionic Pro is expecting the source code, not the compiled versions.

clip_image003

d. Push Package to Octopus

clip_image004

e. Create Octopus Release

clip_image005

f. Deploy the release

clip_image006

Octopus Tentacles

Configure the tentacles (3 on one machine, 1 on another)

To set up Octopus Deploy, you need an multiple environments (in my case SysTest, QA, UAT, and Production). Each environment needs a Tentacle. To simplify this, I use 2 servers and 4 tentacles. In this case, the servers/tentacles will be a temporary place where the code goes on its way to Ionic Pro. So I set up:

  • Server A
    • SysTest tentacle
    • QA Tentacle
    • UAT Tentacle
  • Server B
    • Production Tentacle

Note: I do this to save resources. You could easily use a different server for each tentacle. But the server does so little I decided to conserve resources. I do split dev and prod servers because eventually, the prod code may have values (passwords, etc.) that developers don’t have access to. Since developers don’t have access to “Server B”, it all works out.

Configure Octopus Deploy

My Octopus Deploy project is pretty straight forward. It has one step

clip_image007

There are a few interesting configurations to the step:

Use this setting to update variables in the package.json file.

clip_image008

Use this to transform the config.XML file

clip_image009

I find that Ionic apps aren’t as configurable as I would like. I want lots of values to be configurable. The problem is that I can’t just put a variable placeholder in config.xml. If I did, the XML would not be valid. Consider this snippet of XML from my config file. Note that the #{XXX} syntax represents Octopus Deploy’s variable replacement token syntax. But Octopus only updates my variables during deployments. How can I run this locally?

<plugin name="cordova-plugin-ionic" spec="^4.1.6">
<variable name="APP_ID" value="abcdef" />
<variable name="CHANNEL_NAME" value="#{ChannelName}" />
<variable name="UPDATE_METHOD" value="none" />
<variable name="WARN_DEBUG" value="true" />
<variable name="UPDATE_API" value="https://api.ionicjs.com" />
<variable name="MAX_STORE" value="2" />
</plugin>

The answer is config transformations. I created a second file named config.release.xml. I put the variable tokens inside this new file. As long as I tell Octopus to update variables in my config.release.xml file and also tell it to run config transformations, the values will eventually find their way to my config.xml file after a deployment. This is using the same features used by ASP.NET for web.config transformations. You can read more here.

	<?xml version='1.0' encoding='utf-8'?>
	<widget id="com.YOURDOMAIN.YOURAPP" version="1.0.2" xmlns="http://www.w3.org/ns/widgets" xmlns:cdv="http://cordova.apache.org/ns/1.0" xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
		<plugin name="cordova-plugin-ionic" spec="^4.1.6">
			<variable name="CHANNEL_NAME" value="#{cordova:plugins:cordova-plugin-ionic:CHANNEL_NAME}" xdt:Transform="SetAttributes" xdt:Locator="Match(name)"/>
			<variable name="UPDATE_METHOD" value="#{cordova:plugins:cordova-plugin-ionic:UPDATE_METHOD}" xdt:Transform="SetAttributes" xdt:Locator="Match(name)"/>
		</plugin>
	</widget>

Use this to update variables in other files

clip_image010

Here are the list of Variables:

CHANNEL_NAME and UPDATE_METHOD are used to update the values for Ionic Pro Live Deploy

clip_image011

OK, so at this point, we are set up for our Ionic project to get checked in to source control, automatically packaged up by TFS and then deployed by Octopus. But how do we get it to Ionic Pro? We’ve got a bit more work to do and it involves some more set up on our target servers. Earlier I mentioned 2 target servers called (for this sample) Server A (has SysTest, QA, and UAT tentacles) and Server B (has the prod tentacle). I’ll focus on Server A for now but you can set up Server B the same way.

Security for Ionic Pro with SSH

Since we’ll be using git to push changes to Ionic over SSH, we need an SSH Key. Getting this setup was a little tricky, especially since I don’t have much experience with SSH.

  • First I created a new account in Ionic. I want it to be clear that my code changes are checked in by my CI system, not me. Our account is named DevOps.
  • Create an SSH Key
  • Here’s a trick. Octopus Deploy runs as the System Account. But when you generate the SSH key it will go to a folder for you, the logged in user. Since I was using Git Bash to generate my SSH key, I ran Git Bash as “System Account” with a PSExec command like this:
    • psexec: psexec -s -I gitbash.exe

By doing this, my SSH key will be saved in the correct place.

  • Upload the SSH Key to Ionic for the new user account. In Ionic, go to Account Settings > SSH Keys
  • On the tentacle server you also need to configure the user account that you used to connect to Ionic (the devops account)
    • git config user.email “devops@…”
    • git config user.name “devops”

Server Setup for Git Repos

Remember that on each tentacle, all we are really doing is getting the changed source code from Git and then checking it in to Ionic.

  • On each server, install GIT and install NPM
  • Next, I created a folder C:\IonicStaging. Inside that, a folder per environment:
    • C:\IonicStaging\SysTest
    • C:\IonicStaging\QA
    • Etc.
  • For each folder above, we need to set it up as a git repository by:
    • Navigate to the folder such as C:\IonicStaging\SysTest
    • Execute the GIT clone command for your code. You can clone from your regular Git repo or you can clone directly from your Ionic Pro source as well.
    • Create a local branch for each folder:
      • Example, in the SysTest folder: git checkout -b SysTest
    • Create an Ionic Remote for Git:
      • git remote add ionic ssh://get@git.ionicjs.com:YOUR_USERNAME/YOUR_PROJECT.git
    • Test it out by pushing changes from your local branch to a SysTest branch (it will create it for you) on the remote: Ionic
      • git push ionic SysTest

Setup Channels on Ionic Pro

You will need a channel for each environment

  • Create a channel for each such as SysTest, QA, UAT (Production is there by default)
  • Set the auto-deploy for the channel as such:

clip_image012

PowerShell

All this setup is great but it won’t work without a little PowerShell love. Octopus is pre-configured to run PowerShell scripts during the deployment, if they exist with the correct name. So I have a script named deploy.ps1 inside of my Ionic app. It looks like this:

In the script above, the “#{xxx}” format represents variables that Octopus will replace. You will recall that earlier when I configured Octopus Deploy, I set it to update variables in the file named deploy.ps1.

	$sourceRoot = $pwd
	$env = "#{EnvironmentName}"
	$destinationRoot = "C:\IonicStaging\$($env)\MyMobileApp" 

	$exclude = @('[Content_Types].xml','*.nuspec', '.git', '_rels', 'package', 'deploy.ps1')
	#clean out the target
	Remove-Item $destinationRoot -recurse -Exclude $exclude
	#copy files to target
	Get-ChildItem -Path $sourceRoot |ForEach-Object {
	Copy-Item $_.fullname "$destinationRoot" -Recurse -Force -Exclude $exclude
	}
	Set-Location -Path $destinationRoot
	$gitPull = "git pull ionic $($env)"
	Invoke-Expression $gitPull
	#this will set the npm version in package.json (Octopus.Release.Number will be updated first)
	npm --no-git-tag-version version #{Octopus.Release.Number} --allow-same-version
	#this will set the ionic version based on the npm version
	ionic-version
	git add -A
	git commit -m "#{Octopus.Release.Number}"
	$gitPush = "git push ionic $($env)"
	Invoke-Expression $gitPush

The script does two main things:

First, it updates the version number in package.json using “npm –no-git-tag-version version #{Octopus.Release.Number} –allow-same-version” and you will note that I am setting the version to the Octopus Release Number. It also calls “ionic-version” which copies the package.json version number into config.xml.

The other main function of this script is to get the files into Ionic Pro. The script copies the source files that have been deployed with Octopus (remember, they have not yet been compiled) over to the IonicStaging folder that we created on the tentacle server. Then it does “git add”, “git commit” (using the Octopus Release Number for the comment), and “git push” to Ionic.

Consistent Numbering

Did you notice along the way that I use the same number for my TFS Build Number, Octopus Release Number and Ionic App Version Number (and assembly version number for my .NET apps)? That gives me a great way to trace problems from source code to installed application and back!

Finally Done!

Although each little step is pretty simple, it is fairly complicated to get the whole thing working. To me, it is totally worth it. I love having a pipeline that “just works”:

Check in code –> Code is deployed!

At this point, I’m just hoping I didn’t miss any steps when I wrote this.

 

techbash

TechBash 2016 is a brand new developer conference taking place on September 28-30, 2016. I’m one of the organizers along with a small group of friends from the developer community. TechBash will feature great presentations for you by a variety of awesome speakers from around the country. Our venue, the Kalahari Resort is really cool. It is a brand new Resort, Conference Center, and 100,000 sq. ft. indoor water park! TechBash is located in the Pocono Mountains in PA. That means it’s not only a beautiful setting but it’s close to many people living in the northeastern US. It’s less than a 2 hour drive from New York City, Philadelphia, and northern NJ. And this location is still within reasonable driving distance of many other cities in the northeast!

Our goal is to inspire our attendees to build great things. At TechBash, we are building an environment that is designed for attendees to get the most out of the event. In the many sessions at TechBash you will learn all sorts of new things. But we know that a lot of great things happen outside of the talks as well. That is why we will have a great attendee lounge that will have plenty of space for you to hang out and talk with old friends and new. It will provide a great place for speakers and attendees to have follow up talks after sessions. There will be snacks and coffee. And it will also include our Hack Lab and our vendors as well. The Hack Lab is a space for our attendees, speakers and sponsors to collaborate, brain storm, hack and even demo the results!

TechBash is a community driven, non profit event (although not a charity) organized by the TechBash Foundation.

If you are already hooked, just register (registration info is below)! And please read on if you need a bit more convincing…

Content is Key!

Our goal is to be technology/platform agnostic. We know that nowadays, developers are interested in all kinds of stuff and there is a lot to learn for us all. Having said that, the agenda is weighted towards Microsoft Technology.So while we have content on ASP.Net (including Core), Azure, Xamarin, Visual Studio, C# and more, we also have talks about JavaScript, Docker, Go, Design Patterns, Web Security, OSS, Agile, and more. For all of the latest details, check out the session list. Keep in mind, as with any event, the schedule is subject to change.

Location, Location, Location

imageAs I said above, TechBash is pretty close for many people in the northeastern US. Sure, those of you from NYC to Philly are all set with a simple 2 hour drive. But if you are from DC to Boston to Pittsburg and beyond, this location is a reasonable drive for a 3 day event! Here are some drive times (according to Google Maps – your times may vary Smile). No air travel required!

City Drive Time City Drive Time City Drive Time
NYC 1:48 Philadelphia 1:53 Harrisburg, PA 1:59
Toms River, NJ 2:19 Syracuse, NY 2:28 Albany, NY 2:58
New Haven, Ct 2:49 Baltimore, MD 3:14 Wash. DC 4:01
Providence, RI 4:24 Buffalo, NY 4:43 Boston, MA 4:48
Pittsburgh, PA 4:51

 

Destination: Fun

Ok, so you are headed out for a 3 day tech conference. That doesn’t mean you can’t have some fun too! We’ll fry your brain all day long with tons of great content that you can take back to the office and put to good use. But in the evenings, why not enjoy the 100,000 square foot indoor water park? TechBash is hosted at the brand new Kalahari Resort and Convention Center. Yeah, it’s a state of the art conference center and great hotel. But they have restaurants, a Spa, an arcade, indoor mini-golf, hiking and more. Oh, did I mention the 100,000 square foot indoor water park?

waterparkwaterpark2waterpark3

 

Is this a family event?

Yes and no. We know that other similar conferences have kids’ tracks and family content. Plus, there is the water park. While the venue is certainly family friendly, TechBash won’t have any family content in year one but it is part of our long term plan. Let’s get through one year with content for the “big kids” only. But if you want to bring your family, please do so! We are sure they will have fun!

Sponsorship Opportunities Available

We’d love to work with your company as a partner to help make TechBash 2016 even better. Check out our website and our prospectus for more info and then contact me to get things started.

Want to help out while you are at TechBash?

Yeah, we’ll need some help running this event. If you want to help out, sign up, there is a simple form on the bottom of the website.

The most important part: Registration

Registration is easy, there are two simple steps:

  1. Click here to purchase a conference ticket on EventBrite.
  2. Book a room at the Kalahari. Just call 1-855-356-9208 and mention TechBash to get the best rate.

Thanks for reading. I’ll post more information about TechBash soon.

I had a great time on Friday 3/20 presenting “End To End Development with Schwammy’s Favorite Patterns and Practices”. Thanks to everyone who came out and sat in a packed room for a full day. And even though we had a pretty large spring snow storm, I’m pretty sure everyone stuck around for the whole day. I hope you all made it home safely.

In this talk I covered a lot of content. If you need to follow up with me on any of these topics, feel free to contact me directly or comment on this post.

Thanks for the great feedback you all provided. The response was very positive. There were a few good suggestions and I’ll definitely be taking them into account when we plan a repeat of this presentation.

Here are the files I promised. I hope the solutions all work out for everyone. Please let me know if there are any problems.

By the way… one last tip… I used one of the VS Extensions that I talked about, VSCommands, to create the zip files for download. VS Commands has a feature allowing you to right click in Solution Explorer and Zip up the solution. It will automatically remove source code bindings, ignore bin directories, and more. Very cool.

Note that the sql scripts for the ELMAH database are in the solution in the App_Readme folder.

If you haven’t heard already, there are two new user groups starting up in the Philadelphia Area. Both are hosted on meetup.

ALM

The first is Philly Application Lifecycle Management User Group. The group’s first meeting is Thursday 2/27, 6pm at the Microsoft MTC in Malvern. The topic is “Getting the Most from Team Foundation Server 2013. And the speaker is… ME! I hope you’ll come out and help get this new group started. Here is a summary of the talk:

TFS is a powerful tool for many things including Work Item Tracking, Source Control and more. But what can you do when the standard templates just don’t fit your needs? Every team is different. Did you know that it is actually pretty easy to customize many parts of TFS to fit your team’s process? I’ve recently upgraded an entire IT department’s source control and work item tracking from TFS 2008 to TFS 2013. During the process I customized many facets of TFS. In this talk, I’ll share what I learned along the way. When we are done, you’ll be set to start getting the most from TFS.

Xamarin

The second new group is the Philly Area Xamarin Group. Their first meeting is Tuesday March 11. Looks like the first one is a meet and greet at Field House in Philadelphia. That sounds like fun.

I’m in the process of migrating our TFS Instances to 2013. During my research I’d seen demos with the “Features” feature. In TFS 2013, there is now a Work Item Type named Feature which is meant to be a parent to several Product Backlog Items. I was able to easily create features but I also knew there is supposed to be a Features Backlog. On my TFS cloud instance it works fine but there was no sign of the Features backlog on the web interface for my On Premise instance of TFS 2013. The screenshot shows what I was hoping to find. Continue below to find out how to enable it.

image

It turns out that getting this to show up was really quite simple but not exactly intuitive. Features needs to be enabled as part of the Access Levels portion of TFS.

On the top right side of the TFS web app, click the “gear” to go to the control panel.

image

You may see this menu. If not, skip the next step! If you do, just click the Control Panel link to go to the main control panel page.

image

Hopefully you will see a menu like this:

image

From here, click on “Access Levels” and you’ll see something like this:

image

 

You will see that there are 3 choices for access levels. We’ve got “Standard” set as the default. I’m not positive we’ll keep it this way and every team should decide for themselves what they want to do about the default. However, you must have “Full” Access to see the “Features” backlog. So add some users to that level or set it as the default. Once you do, users with Full Access will see the Features Backlog (similar to the first screenshot in this post).

I’m not exactly sure why the access levels are set up like this. I’m new to TFS2013 so it may be possible to set this visibility in other ways. But it seems odd that enabling the Features backlog is tied to other features like “Team Rooms” and “Test Case Management”. Those are great features but I am not sure why they are all lumped together.