Generating TechDocs for TAP Catalogs

After installing TAP and importing few catalog items, you might have noticed whether View Techdocs options is disabled or even if it is enabled, you see the following error when clicking on View Techdocs options.

In this blog post, I will help you to generate TechDocs and publish them on AWS S3 and finally update tap package to read those docs from S3 bucket.

VMware Official documentation here covers the steps at a high level but i faced several challenges during this process and would like to share those with you here. So, Let’s start.

Pre-requirement for Generating TechDocs

  • A Linux machine with docker installed on it. It can be Mac too but it gave me lot of pain with my Mac M1 and was unable to figure out errors. So I have personally used Ubuntu for this test.
  • Following packages should be installed on the machine you will be using for tech docs generation
    • Node
    • npm
    • npx
  • Following are the versions I have used. This may not exactly match your side but try to have latest version.
ubuntu@dt-vc-ubuntu:~$ npm --version
8.6.0
ubuntu@dt-vc-ubuntu:~$ node --version
v18.0.0
ubuntu@dt-vc-ubuntu:~$ npx --version
8.6.0

Quick Note: I had Node 10.x version on my system and it gave me multiple errors and warning during doc publishing. those error were fixed after updating to latest version.

  • A non root user with enough privileges to execute the tasks. In my case, I used ubuntu user and added it to the following groups. 
$ id ubuntu
uid=1000(ubuntu) gid=1000(ubuntu) groups=1000(ubuntu),0(root),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),113(lxd),995(docker)
  • You are are in the non root user account on your local system where you will be generating tech docs.
  • Docker is up and running
  • Docker image spotify/techdocs is pulled to your local machine
$ docker images
REPOSITORY                                              TAG                            IMAGE ID       CREATED        SIZE
spotify/techdocs                                        v1.0.2                         0f9b27db2e46   2 weeks ago    445MB
  • An AWS S3 Access and S3 bucket created. You can follow VMware official docs here to create a bucket and an account. Couple of points to be aware:
    • When TAP reads S3 bucket for document, it only require read only credentials
    • When you will publish document to S3 bucket, you will need an appropriate permission to write to S3 bucket.
    • You have two options here:
      • You can create separate set of credentials (one read only and one with write permissions) and use them accordingly
      • Or, go with one set of S3 credentials and have enough permissions. I did this one.

Generating TechDocs

Once you have completed above pre-requirements, look for the catalog you have used during TAP installation. e.g. in my case, I have used the following. You might have a different url and if you do not know, refer TAP installation document for the same.

http://github.com/dineshtripathi30/tanzuapp/blob/main/blank/catalog-info.yaml

  • Clone the git repo used for TAP GUI catalog on your local system
git clone http://github.com/dineshtripathi30/tanzuapp.git
Cloning into 'tanzuapp'...
warning: redirecting to https://github.com/dineshtripathi30/tanzuapp.git/
remote: Enumerating objects: 1706, done.
remote: Counting objects: 100% (202/202), done.
remote: Compressing objects: 100% (112/112), done.
remote: Total 1706 (delta 61), reused 202 (delta 61), pack-reused 1504
Receiving objects: 100% (1706/1706), 82.35 MiB | 6.99 MiB/s, done.
Resolving deltas: 100% (1032/1032), done.
  • Navigate to the folder tanzuapp -> blank and notice the files inside it
drwxrwxr-x 3 ubuntu ubuntu 4096 Apr 30 17:26 systems
-rw-rw-r-- 1 ubuntu ubuntu   44 Apr 30 17:26 README.md
-rw-rw-r-- 1 ubuntu ubuntu  134 Apr 30 17:26 mkdocs.yml
drwxrwxr-x 2 ubuntu ubuntu 4096 Apr 30 17:26 groups
drwxrwxr-x 3 ubuntu ubuntu 4096 Apr 30 17:26 domains
drwxrwxr-x 2 ubuntu ubuntu 4096 Apr 30 17:26 docs
-rw-rw-r-- 1 ubuntu ubuntu  341 Apr 30 17:26 catalog-info.yaml
drwxrwxr-x 2 ubuntu ubuntu 4096 Apr 30 17:28 site
drwxrwxr-x 4 ubuntu ubuntu 4096 May  1 09:15 components
  • You have sample mkdocs.yml file here and this will be used during techdoc generation. Also open the catalog-info.yaml file and have a look at it. We will be using the kind: Location , it’s name and namespace during tech docs publishing to S3.
  • Now, We will go ahead and try generating tech docs for tap-gui-catalog-info, you can find this under TAP GUI
  • Run the below command to generate the tech doc
 $ npx @techdocs/cli generate --source-dir /home/ubuntu/techdoc/tanzuapp/blank  --output-dir ./site
Need to install the following packages:
  @techdocs/cli
Ok to proceed? (y) y
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: '@azure/msal-node@1.8.0',
npm WARN EBADENGINE   required: { node: '10 || 12 || 14 || 16' },
npm WARN EBADENGINE   current: { node: 'v18.0.0', npm: '8.6.0' }
npm WARN EBADENGINE }
npm WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated uuid@3.3.2: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated core-js@2.6.12: core-js@<3.4 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Please, upgrade your dependencies to the actual version of core-js.
info: Using source dir /home/ubuntu/techdoc/tanzuapp/blank
info: Will output generated files in /home/ubuntu/site
info: Generating documentation...
info: Successfully generated docs from /home/ubuntu/techdoc/tanzuapp/blank into /home/ubuntu/site using techdocs-container
info: Done!

Note:

  • –source-dir option specifies the location where mkdocs.yml for tap-gui-catalog-info is present. You need to understand this as when you will try generating tech docs for other components, need to specify the correct location.
  • –output-dir is where document will be generated.
  • Once document is generated, you will see the content under ./sites directory
ls -l site/
total 48
-rw-r--r-- 1 ubuntu ubuntu  8322 May  1 08:50 404.html
drwxr-xr-x 2 ubuntu ubuntu  4096 May  1 08:50 add-docs
drwxr-xr-x 5 ubuntu ubuntu  4096 May  1 08:50 assets
-rw-r--r-- 1 ubuntu ubuntu 10087 May  1 08:50 index.html
drwxr-xr-x 2 ubuntu ubuntu  4096 May  1 08:50 search
-rw-r--r-- 1 ubuntu ubuntu   359 May  1 08:50 sitemap.xml
-rw-r--r-- 1 ubuntu ubuntu   195 May  1 08:50 sitemap.xml.gz
-rw-r--r-- 1 ubuntu ubuntu  1820 May  1 08:50 techdocs_metadata.json

Publishing TechDocs

After completing the tech docs generation for a particular component, we need to publish it to S3 bucket. You need to have the AWS Access Key, Secret ID , Bucket Name and region where bucket is created.

  • Export the following environment variables
export AWS_ACCESS_KEY_ID=AWS-ACCESS-KEY-ID
export AWS_SECRET_ACCESS_KEY=AWS-SECRET-ACCESS-KEY
export AWS_REGION=AWS-REGION
  • Once exported, run the following command to publish the tech docs. 
 $ npx @techdocs/cli publish --publisher-type awsS3 --storage-name tapdocs-demo --entity default/location/tap-gui-catalog-info --directory ./site 
info: Creating AWS S3 Bucket publisher for TechDocs
info: Successfully connected to the AWS S3 bucket tapdocs-demo.
info: Successfully uploaded all the generated files for Entity tap-gui-catalog-info. Total number of files: 42
info: Successfully deleted stale files for Entity tap-gui-catalog-info. Total number of files: 0
  • Validate the content on your S3 bucket

Viewing TechDocs on the TAP GUI

Finally, we need to update tap package to start reading the published doc. Let’s do it here.

  • Open up tap-values.yaml file add the following section under tap-gui 
techdocs:
  builder: 'external'
  publisher:
    type: 'awsS3'
    awsS3:
      bucketName: BUCKET-NAME
      credentials:
        accessKeyId: AWS-READONLY-ACCESS-KEY-ID
        secretAccessKey: AWS-READONLY-SECRET-ACCESS-KEY
      region: AWS-REGION
      s3ForcePathStyle: false

Note: Update the bucketName, accessKeyId, secretAccessKey and region values.

  • Once saved, run the following command to update the tap gui package.
tanzu package installed update tap -p tap.tanzu.vmware.com -v 1.1.0 --values-file tap11-values.yaml -n tap-install
| Updating installed package 'tap' 
| Getting package install for 'tap' 
/ Getting package metadata for 'tap.tanzu.vmware.com' 
- Updating secret 'tap-tap-install-values' 
- Updating package install for 'tap' 
| Waiting for 'PackageInstall' reconciliation for 'tap' 

Updated installed package 'tap' in namespace 'tap-install'
  • Once updated, Open the TAP GUI.
  • Click Docs on the left hand side and Click on tap-gui-catalog-info
  • You will notice that it does not fail now, it opens up the document that we published.
  • This is just a sample doc, but you can make it richer by updating respective .md files before generating doc.

Issues I faced during tech doc generation

  • node version was old and during tech doc publishing, following error was shown
CallbackInfo<Nan::ObjectWrap>&)’} to ‘Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<void>&)’} [-Wcast-function-type]
Object.fromEntries is not a function
  • Running npm command with root user fails, this is default behaviour of npm. Thats why, i have requested to create a non root user in pre-req
npx @techdocs/cli generate --source-dir /root/techdoc/tanzuapp/blank  --output-dir ./site
sh: 1: cannot create buildcheck.gypi: Permission denied
  • Lot of the following warnings were observed when node version was old. All gone after node version update.
In file included from ../src/binding.cc:1:
/usr/include/nodejs/src/node.h:573:43: warning: cast between incompatible function types from ‘void (*)(Nan::ADDON_REGISTER_FUNCTION_ARGS_TYPE)’ {aka ‘void (*)(v8::Local<v8::Object>)’} to ‘node::addon_register_func’ {aka ‘void (*)(v8::Local<v8::Object>, v8::Local<v8::Value>, void*)’} [-Wcast-function-type]
  573 |       (node::addon_register_func) (regfunc),
  • Unable to generate doc with non root user due to docker permission denied error. I faced this due to an insufficient permission to non root user. You need to assign an appropriate permission to access docker.
Failed to generate docs from /home/ubuntu/techdoc/tanzuapp/blank into /home/ubuntu/techdoc/tanzuapp/blank/site; caused by Error: This operation requires Docker. Docker does not appear to be available. Docker.ping() failed with; caused by Error: connect EACCES /var/run/docker.sock
  • Tech doc generation failed due to no such image error. To resolve the issue, I pulled that image on my local system by running docker pull spotify/techdocs:v1.0.2 
Failed to generate docs from /home/ubuntu/techdoc/tanzuapp/blank into /home/ubuntu/site; caused by Error: (HTTP code 404) no such container - No such image: spotify/techdocs:v1.0.2

Refer VMware Official doc here

Advertisement

Leave a Reply

Fill in your details below or click an icon to log in:

Gravatar
WordPress.com Logo

You are commenting using your WordPress.com account. ( Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. ( Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. ( Log Out /  Change )

Cancel

Connecting to %s