VCD RoundTable

Hello and welcome to another

wonderful episode of the VCD Roundtable.

This is episode 48, and I atleast have full control of

my computer, microphone, and camera.

Unlike some other people (in the chat), I

just actually solved it on my own.

So with this episode, we are going to look into

Orchestration and Automation,

which is at least when it comes to Cloud

Director, far more a topic for Matthias.

So before we get to Matthias, Toby, any opening words?

No, absolutely not from my side.

Hello and welcome to episode 48.

Yeah, let's hand it over to Matthias

and listen what he is referring to.

Hi guys, this is Matthias.

Yeah, I'm not in control of my mic anymore, because my

MacBook Pro refuses to accept the microphone I configured.

It insists of using my iPhone for whatever reason.

But yeah, nothing I can automate.

Cloud Director Orchestrator and Automation.

Automation has multiple approaches.

For example, we use automation to onboard new customers.

Onboard new customers, for example, is not like a...

We're not automating it to have it

more or to have it onboarded faster.

So from my point of view,

Automation is not always to speed things up.

On the other hand, a big part of

Automation is to have configuration consistency

and there's standardized implementation of something.

We all are human beings, so we are pretty

aware that you just fat finger something.

Maybe you select the wrong allocation model for the OVDC

and then you configure everything.

And then it's like, "Oh, I selected the wrong one."

So delete everything.

So that's a big part of Automation as

well, to have standardized configuration.

I don't know how many guys of you have already played with

Automation on Cloud Director.

So we have two main pillars to automate Cloud Director.

First, we can just use the APIs and build a workflow to do

certain steps, one after the other,

to achieve a goal.

On the other hand, we can use

Cloud Director Event Broker Services

to trigger based on certain events and external systems to

kick off an automation process.

So these are the two main pillars we have.

From my point of view, and I have a big grin on my face

because Yves and Toby both told me,

"Orchestrator is dead" years ago. It's still there.

And it will be there in the future and forever.

So I'm a big "Orchestrator" fan.

On the other hand, I have to admit there is a great PowerCLI

implementation or a cmdlet

for the Cloud Director API as well.

So everybody, feel free to use PowerCLI as well.

I'm more the "Orchestrator" guy,

but that's just my personal opinion.

Most important thing is to automate stuff.

So PowerCLI and Orchestrator can both

be used to access the Cloud Director API

and automate things.

I still stick with the example: tenant onboarding.

Why am I using this example?

Because tenant onboarding includes many, many steps.

So first, guys, what do we need?

We need to select the PVDC.

We need to create an OVDC, select an

allocation model, choose resources.

We deploy an Edge gateway.

We need an OVDC network.

We need IP space allocation, maybe some

default firewall rules in the Edge gateway.

Some users create things like this.

So a ton of stuff which needs

to be done to onboard a new tenant.

That can be done in an automated fashion.

So all these steps I've mentioned can

either be built in PowerCLI or Orchestrator.

Before we dig more into that topic, I need

to make you guys aware that Cloud Director

currently offers two APIs.

We still have the old standard REST-based API.

We still have the old standard REST API.

And the new one is called the Cloud API.

I have to admit, the Cloud API

is the new stuff, its cooler.

I'm refusing to say the better API.

It's an easier API compared to the standard one.

But the Cloud API still misses a lot of important stuff.

Everything about automating and managing virtual machines

and vApps is still not in the Cloud API.

And I don't know when it will be part of the Cloud API.

So what we need to think, if we cover automation, we have

the two main aspects I think we're

going to cover is PowerCLI and Orchestrator.

Power CLI is shipped with its cmdlets for Cloud

Director, and they can be used if

you need to implement something and you don't have a

cmdlet, use PowerShell to implement

the REST API call.

And on the Orchestrator side, it's the exact same thing.

So you could implement the plugin

in Orchestrator and use the plugin.

Honestly, it is nice, though.

I have to admit, it's helpful.

It's nice.

You can solve a few very basic

tasks really fast with the plugin.

It has an implicit authentication,

which you can use in JavaScript code.

So that's pretty nice and straightforward.

But I honestly tend to implement

all the Cloud Director calls via REST.

The plugin has not all the methods and API calls

implemented, so you are dependent on

the development team of the plugin if a

certain feature is available via the plugin.

If not, you need to build it in REST anyway.

So that's the reason I prefer

the REST approach in Orchestrator.

And what I start with, I build a template.

So a few templates that I can use over and over and over,

because I don't want to build

and structure every single workflow to rebuild every single

REST API call over and over and

over.

So it's easy to just grab a template and change a few lines

of code, the method, the URL,

or some inputs, outputs needed, and then use the same

workflow over and over and over.

So it's also a very standardized approach

how to build workflows for automating, in

our case, Cloud Director.

But I use the same mechanism if

I automate NSX or other stuff.

With Cloud Director, as long as we use the Cloud API, we

have a great Swagger documentation

of the API built into the Cloud Director product.

On the top right, just click the

question-- I think it's the question mark.

And then you have a menu entry saying API Explorer.

And then just navigate over there.

Please be aware, Cloud Director

is still a multi-tenant product.

So you have an API documentation

for the tenants and for the provider.

And they are different.

Always keep that in mind.

Oh, what just pops up in my mind is:

There is a very cool mechanism, guys.

If you as a CSP would like to offer your tenants API access

to Cloud Director, there is a new

session to generate user tokens.

So it's somewhere in the UI.

On the top right, where the username is,

you can go to User Preferences, I think.

And there is a section or a menu

entry saying "generate access tokens."

So you can use that access-based

mechanism for the API for your tenants as well.

It's pretty nice.

Also for service accounts and other stuff.

So it gets even more secure.

Nevertheless, so the Cloud API documentation is great.

It's easy to use.

It's built into the product.

You have all the URLs and like a Swagger

documentation is, you have all the tryout,

functionality, copy, paste, etc., all that kind of stuff.

If you have or if you need a REST API call, which is not

part of the Cloud API, you need

to move over to the standard API.

Yes, there is a documentation.

I would phrase it, it's not as easy to use

as the Swagger documentation of the Cloud API.

It's a website in VMware code

and best of luck to find something.

Honestly speaking, I prefer if I need to deal with the

standard API and I have no clue how

a certain API call works.

I prefer using the developer tools of my

favorite browser and just grab the call, the

UI fires in the background against the API because the

Cloud Director UI does or is just

consuming the REST API in the background.

So there is no secret behind that.

Use developer tools, grab the REST API call from the

backend, which is monitored by the

developer tools and then you

have the URL, maybe some data model.

So you can go to the header section, validate which header

it is, which API version it is

and then check the payload.

If it's a put or a post, grab the data model.

Sometimes you will find some hidden options in the data

model if you're using the developer

tools.

So even with the Cloud API, not every

single parameter of the data model is specified.

And then you just use this stuff, you

have sniffed and used a different data model.

It also helps which parameters are mandatory, which are

optional because I, for example,

sometimes struggle with, "oh, I need parameter ABC or where

do I get this information from?"

Maybe it might not be available.

So it's easier to be like, "oh, it's just optional.

I just leave it empty or remove

it from the data model as well."

And then you can do it by REST.

I think that's a pretty straightforward approach for

automating Cloud Director and also basic

tasks.

And then you just move on for NSX and other products.

If you have Cloud Director,

please automate NSX via Cloud Director.

I think that's something very,

it's always like, oh, yeah, thank you.

I saw it in your face, right?

So it is important to say, even if you

automate stuff, stick with the rules.

Cloud Director is the leading system.

If you've Cloud Director and your tenants

are consuming NSX, automate on an NSX via

Cloud Director.

Except, unless you're automating something in NSX, which is

meant to be a managed service

for your customers.

So it would be configured in NSX manually anyway.

One of the features which just comes up

is, for example, Distributed IDS/IPS.

That's a managed service from a CSP perspective.

Okie doke.

So that's basic automation.

The even more interesting part is,

yes, Yves, you want to share some thoughts.

You need to unmute yourself.

Yes, I think the big question for me is, on one end you

said, you don't know when the

Cloud API is going to be finished.

I think the answer to that is very, very straightforward.

It's never going to be finished, because that requires

additional development investments into VCD.

And VMware was very clear that besides patches or anything

else, they are not going to invest

a penny into VCD anymore.

I would not, to be honest, expect anything really coming

back into that one, especially

looking at how long it took them to

get the API to where it currently is.

And I would not expect much to the other ones.

They promised that we will get similar API in the new

systems, but they did not say similar to what.

So whether that is the old-old one, the Cloud API or the

current Automation API, which they

also sometimes refer to.

So the question is, if someone didn't start so far on their

automation journey, what would

be your advice at the moment to

automate anything around Cloud Director?

Would you still advise them to go directly REST?

Do you advise them to use any of the existing packages?

Or maybe use something more like

an infrastructure as code approach?

Yeah.

So I still think the plugin is great, but,

and there's a big but, if you start using

the REST approach, you're more flexible because all the

other products support the REST API

as well.

NSX, the vCenter server, SDDC Manager, even Operations

manager, Operations for Logs,

or it's a bit more hidden, but they have a REST API and

they all use the same mechanism.

If you are familiar dealing with one REST API, you're

familiar dealing with all the

other REST APIs as well.

The only big difference between all

the APIs is authentication, right?

So you once need to figure out how to build

Cloud Director authentication via REST, or

you need to figure out how to build the authentication for

NSX because they're different.

But at least speaking for VMware by Broadcom, the

documentation, how to authenticate against

the REST API is properly written for

every single product I've dealt with so far.

Toby?

Just to add here one thing, especially for those who really

begin to start now with Automation

or playing around Automation, let's call it that way,

because if you really have already

a use case that you would like to automate something, then

maybe it is not the best idea.

But if you're really starting now from scratch and say,

hey, I would like to have people

inside and look and play around with it, I would heavily

recommend to play around with

the REST API of the Operations Manager, especially for the

future, more and more becomes the

centralized API for the whole VCF game.

So if you now really start with the REST API, then I would

highly recommend that you play

around with the API of the Operations Manager.

That's a good mentioning.

Thank you.

So if you're familiar with REST,

you're good to go with all the products.

And it's not related to only VMware by Broadcom.

But I have to admit, I don't know anything about the

documentation quality of REST APIs

of other vendors.

So I'm not commenting.

So that's very useful.

You can even add Orchestrator into the service library of

Cloud Director, and then you can

use a VCD UI to kick off or to start VRO... it's Aria

Automation Orchestrator -- a VRO workflow.

And if you build the input form properly in VRO, it's

identically displayed in Cloud Director.

And if you're really brave, you play a bit with the actions

in Orchestrator to populate drop-down

menus and stuff, and then you can add filters based on

organization ID, which might make sense to have

an input form of a workflow, a multi-tenant based on Cloud

Director, because Cloud Director knows the

organization ID, OVDC user ID, and stuff like this.

And the combination with roles and rights enables the

visibility of certain objects like an OVDC network.

So that's this part.

Let's move to the dark side of

Automation -- event-based stuff.

I think that's even a more challenging area to automate

infrastructure behavior, because

that's exactly what we're talking about.

So within Cloud Director, if you

go--so that's the provider context.

Within the provider context in the administration section,

there is a menu entry called Extensibility.

That's where you add the message broker to the system.

And on the second tab, you can enable events.

And then you can select--you can either trigger that Cloud

Director should send all events to the event bus,

or messaging bus, or to certain events, and you can choose

if it's blocking or non-blocking.

So non-blocking means Cloud Director sends

an event and continues with its workflow,

or if it's blocking, Cloud Director fires the event,

triggers the event, and waits for the response.

So the response could be successful, failed, or cancelled.

And it sits there until it reaches the default time out.

And that's five days.

So you can wait for an external system.

That might make sense if you ramp up a new virtual machine,

and you need to grab an IP address from an external IPAM.

It might make sense to wait for it, because you need to

inject it into the boot process of the guest... for example

And of course, if you use the event-based automation, you

need to implement a RabbitMQ message bus in the backend.

There are some hacks to access the MQTT,

so the internal Cloud Director message bus,

but I'm not the biggest fan of

it, because I didn't understand it.

So it's a bit more of a hack.

So I don't preferr that option, because I leave the

appliances as they are, because from a support perspective.

So I'm not changing anything in the appliance.

Just attach the message bus system, and then you attach

Orchestrator, on the other hand, to the message bus.

Do some configurations to route messages.

So I have a nice blog article out there.

It's like zillions of years old, but still accurate.

And then you just consume the messages, and based on the

message, you start the workflow.

Hint, or today I call it a pro tip.

If you deal with messages, there is something out there

called the VMware Cloud Director Notification Package.

If I have it in my head correctly, I think it was built by

Christophe Decanini initially.

I think, so he's a great guy.

So that notification package enables something very

specific, which is, you can

collect a message from the message bus.

That's Orchestrator out of the box behavior.

But you have a message runner workflow, and the message

runner workflow converts the message

body into Orchestrator usable objects.

Which is very helpful, because

otherwise you need to start dealing with XML.

And dealing with XML in Orchestrator with the Rhino

JavaScript engine is not even a nightmare.

It's like, I don't know. No, it's not working.

Yep, oh, Yves volunteers.

So from now on Yves is doing all the XML stuff for you guys.

I'm not touching Orchestrator

So the notification package.

I have something to write my code now.

I tried it and it can't deal with it because the Rhino

engine in the JavaScript engine

version in Orchestrator is way too old.

And does not understand the XML header fields and stuff.

Which tells me a lot about the product.

It's a great product. It's old. It works.

But coming back to one sentence before, are we sure that

AMQP still is supported in 10.6?

Because MQTT should now be finalized and working. From what

I have heard and what I have tested.

I don't get the access method

and RabbitMQ is still supported.

Does Hock charge an extra license

for RabbitMQ now that he owns it?

Please not.

Just asking.

I don't know. I didn't read anything.

So another very very common use case for the Cloud Director

Notification package is or are the blocking tasks.

If you're using blocking tasks, the

message runner workflow converts automatically.

Or no, let's phrase it differently. The

message contains the blocking task ID.

And the Cloud Direct Notification package, the message

runner workflow converts that ID

into a vCloud blocking task object.

So that can be triggered via the plugin.

So these are some rare cases which

are just the plugin is so useful.

I once built a whole rest mechanism with blocking task ID

to resume or cancel whatever.

But honestly with the plugin, it's just two lines of code.

And then you just trigger the blocking task. It's

successful, continue, it's failed, it's aborted.

And you can also implement based on that behavior some

error handling in the workflow.

Some rollback or whatever is needed.

So that's the event based

automation approach out of Cloud Director.

And the Notification package also enables you to filter the

messages based on organizations, based on users, based on

event types or whatever is needed.

Is part of that notification package.

The only thing is, the message runner workflow is not

working anymore out of the box.

Second pro tip.

The methods of the date time object have changed.

You just need to go into the code, it's around line 140,

and just change three or four methods.

And then it's all good. You just need to

alter the workflow and then it's working again.

It tells the error message tells

you exactly what's not working.

Just validate that the methods are wrong, use the correct

methods and you're good to go.

That's the second pro tip.

Yeah, so that's...

So maybe one last idea based on a statement of Toby.

Because Toby said, and that's

something you always should keep in mind.

If you're new to the automation game and you want to start

with automating stuff, I have two tips.

First, never ever just sit in front of the computer and

tell yourself, "I'm now going to automate something."

It's not working.

It's not working, because you don't have a goal. You cannot

achieve the goal. You will never have a result.

Always come up with the use case and implement the use case

and please start with very simple ones.

And that's my second point.

Never ever start, if you're new to the game, start

automating virtual machines or vApps in Cloud Director.

That also refers back to the vCenter server. Please do not

start automating virtual machines.

Because these are from the amount of methods and properties

they have, they are by far the most complex objects.

Just start with simple tasks. Let's call it that way.

Yes, exactly. Toby, you're completely right.

If you're struggling in the beginning, what could be an

easy thing to automate just to play around with it?

Create an Edge Gateway in an existing org.

The OVDC itself is a very complex object

as well because of the allocation model.

That's the only reason. And I have

another, I today have a ton of pro clips.

If you would like to build an OVDC with Flex as an

allocation model, the REST API is the only way to go

forward, because the plugin is

not aware of that allocation model.

It's that simple.

It was just introduced about five years ago.

Yeah, so it's like it never went to school.

Good. I think that is a good starting

summary around Automation Orchestration.

If you have any questions in any of these areas, feel free

to ping the master of old-school automation called Matthias.

No, his name is Sascha.

Well, Sascha does the fancy new stuff.

And also, if you have any other topic suggestions, feel

free to drop them somewhere in the show notes, no matter

which platform you are using, so that we get that.

Do we know already the topic for

the next recording in two weeks?

Okay, I see silence.

Okay, I see silence. So that means you should better follow

us on social media and see what we are going to announce,

what is going to be Episode 49.

And then we better have a good

idea of what's going to be Episode 50.

But nevertheless, I would say

thank you all for listening in.

You can find this also on any of the

podcast platforms and subscribe over there.

Any closing words, Toby?

Not from my side. Start with automation

as soon as possible and have a happy day.

Yeah, guys, go into automation. It's just cool. It's better

than just clicking and watching

progress bars moving and stuff.

And then you free up time to learn

new topics and do even cooler stuff.

The first steps are pretty hard to

take, but get used to it, go into it.

It's not that difficult once you get used to the mechanism.

I think, and I will end with a side take, I think it was

William Lam who said, "Before I do

it once, manually, I automate it."

A good man, a wise man.

Thanks for that closing statement. See you all on one of

the next episodes and automate the world.

Thanks for tuning in, guys. Bye.