P3.NET

Using T4 to Create an AppSettings Wrapper, Part 7

In this final article in the series we’re finishing up the deployment projects started last time. When we’re complete we’ll have a set up projects that can be used to build and deploy T4 templates where needed. The projects will be able to support any number of templates so maintenance will be simple even as more templates are added. In the previous article we moved the core template functionality into a library that we can expand upon. We also set up an item template project to store the root T4 template that someone would use. In this article we’re going to create the project to store the nested templates (we’ll discuss why later) along with the Visual Studio Extension (vsix) file that will be used to install everything.

Update 28 Aug 2016: Refer to this link for updated code supporting Visual Studio 2015.

Corrections

There are a couple of corrections that need to be made from the source in the last article to prepare for the setup project.

In the AppSettingsTemplate.tt file the assembly reference for the shared assembly needs to include the .dll otherwise the T4 host will assume it is coming from the GAC. An assembly reference for EnvDTE is also missing so it needs to be added.

<#@ assembly name="TemplateLib.dll" #>
<#@ assembly name="EnvDTE" #>

In the nested template there were still some hard-coded references to the original class name rather than using the ClassName property in the template. Do the replacement so the template behaves properly based upon the name that is ultimately used.

Nested Template Project

For the nested templates we need to create a new VS Project Template project. When a developer uses one of the item templates they will only add the core template to their project. The nested template, and support assembly, will need to be stored in a shared location that the T4 host can find. The easiest way to do this is to use a VS Project Template project. All the shared templates will be stored in this project. We could technically even store the base template class code in this project but I find it easier to keep them separate.

Add a new project to the solution (Visual C#ExtensibilityC# Project Template) called TextTemplates (the name is not really relevant). Once the project has been added remove all the files from the project as they will not be needed. Add a reference to the shared assembly project that was created earlier.

This project mirrors the item template project so set up the same folder structure for each template as needed. The nested template file will be moved to this project. Every time a new nested template is added the following steps need to be followed.

  1. Create a subfolder in the project
  2. Add the nested .tt file to the subfolder
  3. For each .tt file set the following properties for the item
    • Build Action = Content
    • Copy to Output = Copy Always
    • Custom Tool = (blank)
    • Include in VSIX = True

There is only one issue with the approach we’re taking – T4 does not know where to find our nested templates and custom assembly. Therefore we need to update the search path that T4 uses to include the installation path of the package. The simplest way to do that is to add it to the package definition (.pkgdef) file. When a package is installed the package definition is processed to allow the package to do any customization it needs. We can use this feature to update the T4 include path.

Add a new text file to the project. It’s name must match the name of the project and it should have an extension of .pkgdef (i.e. TextTemplates.pkgdef). Change the following properties for the item.

  • Build Action = Content
  • Copy to Output = Copy Always
  • Include in VSIX = True

When T4 runs across an assembly or include directive that it cannot find then it uses the registry to search for additional paths. Each time we add a new template we need to add the path to the template to the list using the .pkgdef file. Note that the subfolder we use in the project will correspond to the subfolder that the template is installed to so we need to include the full path. Here’s the code we’ll add to the package definition file.

[$RootKey$TextTemplatingIncludeFolders.tt]
"IncludeMyAppSettings"="$PackageFolder$TextTemplatesAppSettings"

This will add a new key to the registry with the given name and value. The installer will replace $PackageFolder$ with the installation path for the package. The project name follows that and the last name is the folder that was used when adding the template to the project. It is critical that the key name start with “Include” otherwise the T4 host will ignore it. It must also be unique. Each new subfolder of nested templates will need a corresponding name-value pair in the definition file.

Some discussion of this can be found in MSDN. But credit for pointing me in the correct direction when I was trying to figure this out has to go here.

VSIX Project

We’re on the home stretch. We have set up the project for the shared assembly code, defined the item templates that will be available to the developers and got the support templates hooked up to T4. The final step is to create a VSIX file to install everything. By far this is the most frustrating part because VSIX is very picky about how things have to work and it can be a bear to work with.

Create a new VSIX Project (Visual C#ExtensibilityVSIX Project) called MyTemplateSetup. The manifest editor will open up. The manifest controls several important aspects of the setup including the information the user sees, the files to be installed and the version of the setup.

  1. Set Product Name to the name you want to the extension to appear as in the gallery.
  2. The Product ID must be unique and should already be set.
  3. Set Author to an appropriate value.
  4. The Version should default to 1.0.
  5. Provide a description of the extension.
  6. Since this is a template installer the Tags should probably be set to Templates.
  7. Optionally set the other attributes such as licensing, icon and release notes.

Switch to the Install Targets tab. This tab indicates what version(s) and edition(s) of Visual Studio the extension supports. The default is Visual Studio 2012 Pro or higher which should be fine. Microsoft licensing does not allow third-party extensions to the Express products. If a newer version of Visual Studio comes out then the version number can be updated.

Switch to the Assets tab. This is where we identify the files to be installed. This is also where things can get difficult if the projects were not created using the right project type.

Add a new asset for the shared assembly.

  1. Type is Microsoft.VisualStudio.Assembly
  2. Source is a project in the solution
  3. Project will be the shared assembly project
  4. Click OK

Add a new asset for the item templates.

  1. Type is Microsoft.VisualStudio.ItemTemplate
  2. Source is a project in the solution
  3. Project will be the item template project
  4. Click OK

Notice that the path that is displayed includes an output group. This property is set inside the item template project. If the referenced project actually isn’t an item template then the build will fail.

Add a new asset for the nested templates.

  1. Type is Microsoft.VisualStudio.VsPackage
  2. Source is a project in the solution
  3. Project will be the text template project
  4. Click OK

An interesting (and sometimes problematic) thing that happens is that project assets are added as references to the project. For the nested templates this is problematic because by default the setup project expects an assembly to be generated. For the nested templates there is no assembly so open the properties for the nested template reference and set Reference Output Assembly to False. If this is not done then a compilation error will occur.

Switch to the Dependencies tab. This is where any dependencies can be defined. The .NET framework is already included but we depend upon T4 Toolbox so that needs to be added as well. Assuming it is already installed on the machine you can do the following. to add the dependency.

  1. Source is Installed Extension
  2. Name is T4 Toolbox
  3. Version range will be set to the current version but you can adjust this as needed
  4. Click OK

One issue with dependencies is that if the user does not have the dependency installed they may get a generic error message before they get information about the missing dependency. Hence it is useful to put dependencies in the description of the extension.

Almost done. The last thing we need to do is add another package definition file (.pkgdef) matching the name of the shared assembly project. As was done earlier, configure the item properties.

  • Build Action = Content
  • Copy to Output = Copy Always
  • Include in VSIX = True

The package definition file used earlier had an entry for each template. This file will contain an entry for the shared assembly.

[$RootKey$TextTemplatingIncludeFolders.tt]
"IncludeMyTemplateAssemblies"="$PackageFolder$"

Compilation

Time to compile the setup but before you do I recommend that you change the VSIX properties in the setup project to not deploy to the experimental instance of VS. The experimental instance of VS is designed to allow you to test your packages without impacting your main development environment. Unfortunately in my experience it does not work well when you have additional packages or extensions installed. You end up sitting through lots of error dialogs.

If you do not use the experimental instance of VS then you won’t be able to easily debug your template. However there is a simple approach that I find useful. In most cases you will develop your template in a stand alone project so you can tweak the template. Once it is added to the extension though you can still change it by finding the directory where the extension was installed. By default it will be a randomly generated directory under <VSDir>Common7IDEExtensions for shared extensions or <profiledir>AppDataLocalMicrosoftVisualStudio11.0Extensions for user extensions. Once you find the directory you can edit and/or replace the files until you’ve resolved any issues you’re having. You can then update the project and redeploy.

To test the setup simply run the .vsix file. Start VS, create or open a project and add the new item template to the project. Confirm the generated code is correct. In general if you are adding or removing extensions you should ensure that all instances of VS are closed first. VS doesn’t full install/remove extensions until it is restarted and having multiple instances running can cause problems with that.

Versioning

Versioning of the extension is incredibly important. When VS is looking for an updated extension it uses the version as defined in the manifest. Ensure that whenever you build a new version of the setup that you increment the version number. You should also consider keeping the major/minor version number of the shared assembly in sync with the manifest file.

Do not change the product ID. If you do then this becomes a brand new extension that can be installed side-by-side with the old one. In most cases this can cause conflicts that you would want to avoid.

Troubleshooting

Troubleshooting T4 templates is an art more than a science but here’s a few thoughts based upon my experience.

If you get a compilation error while compiling the setup project saying it cannot find the assembly for the nested templates project then you forgot to change the reference’s properties. Refer to the earlier section on fixing that.

If the root template cannot find the nested template then the registry was not updated properly. This can occur if the extension was installed but VS was not restarted, multiple instances of VS were running or if something went wrong. Uninstall the extension, restart VS to clean everything up, shut down VS and reinstall the extension.

If the shared assembly file cannot be found then the above comments about the root template also apply. Additionally ensure that the assembly reference in the nested template includes the .dll or the assembly is in the GAC.

If the generated file contains an error token then use Error List to determine the actual error(s) that occurred. In most cases it is a compilation issue with the nested template. Load up the nested template in VS and edit it until it compiles. Then rebuild the deployment package.

If the item templates do not show up then ensure that they were properly added to the setup project and that they appear in the extension directory. The path to the extension directory is contained in the install log that is accessible at the end of the extension installation.

When adding a new template get it working in a standalone console application and then move it into the templates solution. This will make it easier to make changes and you can still rely on the shared assembly and break up the template into the item and nested components.

Enhancements

There are quite a few enhancements that could be made with all this code. One area that I personally recommend is using data annotations for validating configuration properties. Apply annotations to the configuration properties where appropriate. Alternatively implement IValidatableObject in the template class. Then modify the Validate method in the base class to validate the annotations as part of the core validation. This eliminates the need for per-template validation of things like required parameters.

Another area of improvement is breaking out all the non-template functionality. I personally like extension methods so you could move all the non-template functionality into extension methods.

Yet another area is defining some common template generation methods that derived types can override. For example most T4-generated code should be marked as non user code so they won’t step into it. Wrapping this up in a method allows derived types to call it where appropriate and, optionally, add their own attributes.

For deployment you can provide the developers with the VSIX package. If you’ve read my earlier article on hosting your own private VS extension gallery then you could deploy the VSIX to that as well.

Download the Code

Comments

  1. Joe Tozzi

    Michael,

    I was trying to follow along with your article, but as soon as I add the pkgdef file and debug with experimental instance, my ability to debug goes away as Visual Studio reports that the breakpoint will not be hit, no symbols loaded.

    The other problem I am having is that when I run without the pkgdef file, I get the following exception:

    There was an error loading the include file ‘T4Toolbox.tt’. The transformation will not be run. The following Exception was thrown:
    System.UriFormatException: Invalid URI: The format of the URI could not be determined.
    at System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind)
    at System.Uri..ctor(String uriString)
    at Microsoft.VisualStudio.TextTemplating.VSHost.TextTemplatingService.CheckSecurityZone(String path)
    at Microsoft.VisualStudio.TextTemplating.VSHost.TextTemplatingService.LoadIncludeText(String requestFileName, String& content, String& location)
    at Microsoft.VisualStudio.TextTemplating.Engine.ProcessIncludeDirective(Directive directive, ITextTemplatingEngineHost host, VisitedFiles includedFiles)

    I have even tried to add a reference to T4Toolbox to my VSIX project in hopes that including the dll in the output directory would allow the package to know where the include file was.

  2. Are you targeting VS 2015? If so then you should refer to the updated post I created for getting this stuff to work with VS 2015. There were changes to the pkgdef file for VS 2015. This article was originally written for the VS2013 preview so I also posted an update to some changes that had to be made for the RTM version.

    Once you’ve made the appropriate changes then try again. Note that, as I mentioned in this article, I have mixed results when using the experimental VS instance. If you do decided to go that route then you need to ensure that you install T4 Toolbox in the experimental instance. The experimental instance is a sandbox and won’t have the extensions installed that you may have in your regular copy of VS. You have to explicitly install the one’s you care about. Additionally you are probably going to get lots of errors about packages not loading. That is because VS relies on a lot of stuff that won’t be available in the experimental instance. You can ignore these but it does cause a significant slowdown when trying to debug.

    1. Joe Tozzi

      Thanks for the quick reply. I did read the updated post but mistakenly left a comment in this one.

      One thing I noticed when running the experimental instance was that all downloaded extensions were disabled. However even after enabling T4Toolbox it I still get the error

      “There was an error loading the include file ‘T4Toolbox.tt’. The transformation will not be run. The following Exception was thrown:
      System.UriFormatException: Invalid URI: The format of the URI could not be determined.”

      1. That error does seem a little odd, like the path you’re using in your #include is incorrect. Can you post that?
        Also, the primary purpose of the .pkgdef file is to put the path to the .tt include files in the registry. This is what allows the generator to actually find them. Can you verify that the registry entries are correct? You should have a couple of entries for your VSIX package.

        For a locally installed extension it’ll be HKCU\Software\Microsoft\VisualStudio\14.0_Config\TextTemplating\IncludeFolders\.tt. If you’re using the experimental instance then the 14.0_Config will have a number at the end to distinguish it from the normal instance.

  3. Joe Tozzi

    Odd indeed considering that every example I look at that uses T4Toolbox just has an include to with no specific path. Additionally in the VSIX source.extension.manifest I added a Dependency on T4 Toolbox for Visual Studio 2015. What is this for if not to inform the package of the necessary third-party extension needed for execution?

    As far as the pkgdef file is concerned, when it was included in the project, I did see the correct path in the registry. However as I stated before, with it’s inclusion, I could no longer debug the project so I removed it. When I run the program now without it there is no registry key that includes the Product ID of my VSIX package.

  4. Joe Tozzi

    And now it gets really bizarre. I went ahead and re-added the pkgdef file to my project, rebuilt it and ran it. This time the experimental instance started and my breakpoint was still valid. However, there is still no key in the registry with my Product Id with the correct path.

    My pkgdef file is as follows:

    [$RootKey$\TextTemplating\IncludeFolders]
    [$RootKey$\TextTemplating\IncludeFolders\.tt]
    “IncludeTemplates”=”$PackageFolder$\Templates”

    1. For your .pkgdef file itself the registry keys don’t look correct. You have a $\ but it should only be $. Take a look at the Modifying the Solution section of the updated article. If that doesn’t solve it then there are some additional things to check.

      The manifest file is used for installing VSIX packages. When you add a dependency there it tells the installer to not allow installation if the given package is not already installed. It doesn’t actually do anything beyond that.

      If the registry isn’t getting updated then there could be a couple of reasons. The first thing I’d verify is that you’re looking in the correct key. If your VSIX is marked to install for all users then the entry won’t be under HKCU as it applies to the entire machine. This would trigger a UAC prompt during installation. If your VSIX is installed per user then it’ll be in the HKCU hive. Remember that for the experimental instance the actual subkey is going to be something like 14.0_Config_??? as each instance gets its own registry subkey.

      Did you verify you had marked the .pkgdef file with the appropriate build and copy actions and include it in the VSIX? A VSIX is just a zip file so open it up using your favorite tool and verify the .pkgdef file is in there. Remember that the name of the file must exactly match the project’s name otherwise it won’t be found during installation.

  5. Joe Tozzi

    I did forget to change the copy output & include in VSIX for the pkgdef. As far as the paths are concerned, your article lists the directory structure without backslashes like this…

    [$RootKey$TextTemplating]
    [$RootKey$TextTemplatingIncludeFolders]
    [$RootKey$TextTemplatingIncludeFolders.tt]
    “IncludeP3NetAppSettings”=”$PackageFolder$TextTemplatesAppSettings”
    “IncludeP3NetServiceClient”=”$PackageFolder$TextTemplatesServiceClient”

    However the actual pkgdef file in your source code has them like this…

    [$RootKey$\TextTemplating]
    [$RootKey$\TextTemplating\IncludeFolders]
    [$RootKey$\TextTemplating\IncludeFolders\.tt]
    “IncludeP3NetAppSettings”=”$PackageFolder$\TextTemplates\AppSettings”
    “IncludeP3NetServiceClient”=”$PackageFolder$\TextTemplates\ServiceClient”

    Can you verify which is correct?

    1. Taking a look at the working pkgdef that I have, yes the slashes should be in their. It looks like the HTML editor pulled them out of the article.

      Looking at the VSIX file it does appear that the T4Toolbox.dll is in the file. That is likely because in order to compile the shared assembly that relies on T4T I had to add a reference to it. Since that assembly is not in the GAC when the setup file added a reference to the project it brought in the assembly as well. Since the T4Toolbox assembly is part of the VSIX then when it gets deployed the dll would get deployed along with the shared assembly that was defined. The path to the shared assembly is registered using its own .pkgdef file. So you should have 2 of them. The assembly .pkgdef file is similar expect it has a single key-value pair that points to $PackageFolder$ so that T4 can find the assemblies. Do you have both of these files?

      1. Joe Tozzi

        I figured the editor had removed the backslashes so that is why I looked at your file.

        Not sure I understand what you mean by having 2 pkgdef files. I don’t really have an assembly that relies on T4Toolbox to compile. I simply added the reference to T4Toolbox in my project thinking that was why it couldn’t find it when running the transform, but I am still getting the UriException. Currently when I open up the VSIX file I see the following files:

        Resources folder (contains package image)
        Templates folder (contains all template files)
        [Content_Types].xml
        extension.vsixmanifest
        Humanizer.dll
        Scaffolding.Package.dll
        Scaffolding.Package.pdb
        Scaffolding.Package.pkgdef
        T4Toolbox.dll

      2. Joe Tozzi

        Ok, once I configured the pkgdef file to be copied to output and included in VSIX when I run the program, the experimental instance comes up showing my VSIX package and t4toolbox as installed & enabled. I can now see the correct IncludeTemplates key within the registry pointing to the templates directory under my project in localappdata. However, my breakpoint says it will not currently be hit. No symbols loaded for this document. Additionally, my customized menu option is no longer appearing under the Tools menu.

        1. Are you trying to get a regular VS extension working in the same VSIX as some custom item templates you have defined? The article I wrote was using T4 Toolbox and extending it to provide additional functionality. If you aren’t doing that then you don’t really need to include T4 Toolbox at all. You would just need to ensure that your .tt files are in the include path for T4.

          As for your inability to debug your non-template stuff I cannot really say what is going on there. Historically if you made changes to your addin you’d have to tell VS to reset the addin otherwise it wouldn’t re-register it. I’m not sure if that is still true with the new extension model. You might need to post this question on the VS Integrate forums. It is also possible your extension simply hasn’t been loaded yet. VS won’t necessarily load your extension until it is first used. As such you’d have to trigger your extension before it would run, unless you are debugging in another instance of VS, in that case the debugger should trigger the loading of the assembly but you could verify using the Modules window. However if you were showing a custom menu item before it should still show up which leads me to believe you might need to uninstall and reinstall the extension. Note that uninstalling an extension doesn’t actually do anything in the current session. You have to restart VS before it will uninstall. So if you uninstall, install and restart you can run into issues. You should instead uninstall, restart VS and then install.

          1. Joe Tozzi

            No I am simply using a VS extension to provide a menu option to show an input form that will then invoke a text transform. It should not be this complicated. The only reason I need T4Toolbox is because it supports outputting multiple files.

            If you have some time and send me your email, I can send you the project. But basically I have a main template that transforms other templates to create a bunch of files in various projects & folders. When I run the custom tool on the template, it transforms the files and adds them to the folder structure that I have locally in the solution (which is the same structure of the project that would run the extension). But for whatever reason I get the UriException for T4Toolbox.tt when running it from the extension.

            The whole reason for using the experimental instance is so that your extension gets installed/loaded automatically and you can debug your program. I did try and reset the experimental instance to see if that made a difference and unfortunately it did not.