Stuff I'm Up To

Technical Ramblings

VCSA root Locked Out! — June 7, 2019

VCSA root Locked Out!

This gave me cause for tears today. The VCSA (vCenter Server Appliance) management Web UI (https://vcsa:5480) decided not to let me in as root. I’m guessing I spannered the password a few too many times.

It’s a very good job that at some point in the past I put my public key onto the system so I could use my plain old no password required private key to logon to the system using ssh!

SSH Logon with Private Key

Now I’m logged onto the console how do I go about getting access back to the Web UI? I discovered that the VCSA system uses pam_tally2 to lockout sessions. What I needed to do was reset the root account:

# pam_tally2 --user=root
 Login           Failures Latest failure     From
 root               10    06/07/19 14:12:11  unknown
# pam_tally2 --user=root --reset
 Login           Failures Latest failure     From
 root               10    06/07/19 14:12:11  unknown
# pam_tally2 --user=root
 Login           Failures Latest failure     From

Now I can logon to the Web UI!

The lesson to learn here is to install your public key onto your precious Linux boxes!

Advertisements
Spectre and VMWare — March 8, 2019

Spectre and VMWare

For some time we’ve suffered a problem with our Windows 7 VDI systems that has prevented us from applying Windows Updates.

If we applied any of the rollups from March 2018 onward the VDI session would reboot itself under one special condition. If a user/client used the Cisco AnyConnect VPN software within the VDI Guest then almost exactly 2 minutes and 10 seconds after connecting, the VDI machine would throw a fatal error and reboot. Instantly terminating the users session.

Continue reading
Cross Origin Resource Sharing and Content Security Policy — December 4, 2018

Cross Origin Resource Sharing and Content Security Policy

Got to love having a vendor carrying out half a job… again.

Having installed a new VMWare Horizon environment for Windows 10, I thought we’d at least have Blast available via HTML  – which we don’t currently have in our Windows 7 Horizon setup.

During the install I setup a load balancer which only really handles the authentication process. This worked fine using IE or Edge, at which point I guess the vendor decided that’s enough testing and it’s considered functional. After they left I fired up my Chrome browser and found it didn’t work. So I tried Firefox with the same non-functional result.

Checking the console log in Firefox I see:

Content Security Policy: The page's settings blocked the loading of a resource at wss://192.168.61.12:22443/d/36BC344E-DAD5-4EA5-A44C-12456F74432D/?vauth=LaQJrs2RppeiZGX9gOtj75vekprtuEDcgD2C6tba ("default-src").

A trawl of VMWare documentation results in: https://docs.vmware.com/en/VMware-Horizon-7/7.6/horizon-security/GUID-FD679D1D-E037-4EDF-A96F-F0CD85FFE724.html

Now all I have to do is translate that to Nginx so I can put that into the config.

Editing my ssl/snippets.conf file and changing the CSP header, I added the missing parts for wss: and blob: to end up with:

add_header Content-Security-Policy "default-src 'self' wss:; script-src 'self' 'unsafe-inline' 'unsafe-eval'; img-src 'self'; style-src 'self' 'unsafe-inline'; font-src 'self'; frame-src https://horizon.domain.tld blob:; object-src 'none' blob:; connect-src 'self' wss:; child-src 'self' blob:;";

A reload of Nginx and a refresh/reload on the browser and I’m into the Horizon Desktop!

References

https://docs.vmware.com/en/VMware-Horizon-7/7.6/horizon-security/GUID-94DAC7B8-70A3-4A91-8E70-2B2591B82866.html

https://www.owasp.org/index.php/Content_Security_Policy_Cheat_Sheet

ESXi 6.0 to 6.5 Upgrade — October 14, 2018

ESXi 6.0 to 6.5 Upgrade

This weekend has turned out to be a challenge. Upgrading our VMware Horizon 7 estate to the latest release involved upgrading all the components from connection servers, security server, composer, vCenter and vSphere hosts.

Last weekend was upgrading the connection servers, security server and composer. This weekend is vCenter and the vSphere hosts.

99.9% of the skills required are really about how strong your Google Fu is!

hongkongphoeey

My skills include Google-fu and Duck-Jitsu, but I’m a Bing-do novice!

Continue reading

VMware Remote Console for Linux — November 22, 2017

VMware Remote Console for Linux

This has frustrated me for as long as I can remember. How do I manage our VMware vSphere estate when the tools provided don’t work reliably on Linux?

First there was the vCenter problem using Flash Player. Thank fully they release v6.5 which has a new HTML5 based interface – no more Flash Player!

https://vcsa/ui

Then inside there you could download the VMware Remote Console (VMRC) and isntall that to allow you to remote onto the actual vSphere guest and not rely on other Guest remote tools like RDS or VNC.

Only trouble with VMRC is that it would not install on my Debian system. I upgraded to the Debian Buster/Sid (testing) version and still can’t get it to work.

Then I couldn’t uninstall it either!

The uninstall complains that there is an unmet dependency for vmware-usbabitrator<=17.1.1. Try as I might I couldn’t get that to install either. I ran the installer bundle with a -x [path] to extract it then manually tried to get the vmware-usbarbitrator to run. Then gave up.

Time to resort to using VMware Workstation Player! Yes, the player can open vmrc:// links. But I couldn’t get it to install because it too complained about vmware-usbarbitrator. So I had to revisit removing VMRC.

To get the removal to work I used DB Explorer for SQLite and opened the /etc/vmware-installer/database file. Then deleted the row from the table component_dependencies that contained vmware-usbarbitrator>=17.1.1

Selection_005

Then I could remove VMRC using:

$ sudo vmware-installer -u vmware-vmrc

This did the trick and it got rid of VMRC. A vmware-installer -l still showed VIX so I removed that too.

$ sudo vmware-installer -u vmware-vix

Now my WMware Workstation Player bundle installed successfully. So I ran it from the menu. I left the license empty at this point and continued to accept the dialogs required to get to the main VMware Player app.

Now it’s just a case of going back to my vCenter Server Appliance (https://vcsa/ui) management interface and clicking on a Guests “Launch Remote Console” link. It fires up VMware Player and asks for credentials for the vcsa and up pops the guest remote screen!

 

vSphere SSH failed to connect to host — November 17, 2017

vSphere SSH failed to connect to host

When trying to apply patches to one of our ESXi 6.0 hosts I found I couldn’t connect to it using ssh. Stopping and starting SSH from vCenter didn’t work. Neither did disabling/enabling from the DCUI.

From my client I’d see:

ssh_exchange_identification: Connection closed by remote host

So then I resorted to checking out the server from the console. First make sure I stopped SSH from either of the GUI’s.

Use ALT-F1 at the DCUI and logon to the host using your root account.

Then I tried to start sshd as a daemon using:

# /usr/lib/vmware/openssh/bin/sshd -D

Which reported errors Unsupported option running and Unsupported option PrintLastLog

So I editted my /etc/ssh/sshd_config file. Don’t know what caused it. But it was just a # missing from the first line. I guess I must have spannered it at some point when editing it to disable some ciphers. But the good news is using this method I can at least get some clear output from sshd -D to tell me why it wasn’t starting properly.

# running from inetd
# Port 2200
Protocol 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key

UsePrivilegeSeparation no

SyslogFacility auth
LogLevel info

PermitRootLogin yes

PrintMotd yes
PrintLastLog no

TCPKeepAlive yes

X11Forwarding no

...

So just to be safe I checked the other hosts and copied an sshd_config from one of the known good ones.

Horizon View Client v4.6.0 — November 15, 2017

Horizon View Client v4.6.0

I decided to upgrade my VMware Horizon View client today. It still has the same kind of issues as detailed here: https://warlord0blog.wordpress.com/2016/10/21/vmware-horizon-client-for-linux/

This time around my problems were with libgstreamer components. Even though I ensured they were installed the libraries were a different version that required by the client.

Specifically required:

  • libgstapp-0.10.so.0
  • libgstbase-0.10.so.0
  • libgstreamer-0.10.so.0

On my Debian Stretch install I had 1.0 versions.

So a quick fix by linking these made the scan issues go away.

$ cd /usr/lib/x86_64-linux-gnu
$ sudo ln -s libgstapp-1.0.so.0 libgstapp-0.10.so.0
$ sudo ln -s libgstbase-1.0.so.0 libgstbase-0.10.so.0
$ sudo ln -s libgstreamer-1.0.so.0 libgstreamer-0.10.so.0

Continue reading

Horizon Client Stealing my Mouse — September 27, 2017

Horizon Client Stealing my Mouse

On my Linux VMWare Horizon client (v4.5.0 5650368) it doesn’t seem to matter what choice I make about NOT Connecting USB Devices at Startup it still continued to take over my Logitech USB Receiver.

I’d have to use the keyboard and navigate the menu so I could get control of my mouse back. Thankfully I don’t have a Logitech keyboard that uses the same receiver.

It was an easy fix, but I don’t know why it does it. The permissions to the ~/.vmware folder and files all seem OK. IT’s an easy fix of just editing the file view-preferences and amending the line or lines as follows.

$ vi ~/.vmware/view-preferences
...
view.usbAutoConnectAtStartUp = "FALSE"
view.usbAutoConnectOnInsert = "FALSE"
...

 

VMWare Restart Guest from Command Line — August 16, 2017

VMWare Restart Guest from Command Line

We don’t have to do this so often. So when we do I always forget the syntax.

Login as root on the host of the guest OS. Find the numeric VMID of the guest and issue a power off/on command.

# vim-cmd vmsvc/getallvms | grep -i "[GUESTNAME]"
Vmid                   Name                                                             File                                                   Guest OS          Version                                                                                                      Annotation                                                                                                   
114    PaymentsTest                            [Datastore-1] PaymentsTest/PaymentsTest.vmx                                               windows8Server64Guest   vmx-10

# vim-cmd vmsvc/power.getstate 114
Retrieved runtime info
Powered on

# vim-cmd vmsvc/power.off 114
Powering off VM:

# vim-cmd vmsvc/power.getstate 114
Retrieved runtime info
Powered off

# vim-cmd vmsvc/power.on 114
Powering on VM:

 

References: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1038043

VCSA 6.0 U3b to 6.5 U1 — August 10, 2017

VCSA 6.0 U3b to 6.5 U1

So far this upgrade seems to frustrating straight out of the box! We already run a VCSA (vCenter Server Appliance) and the process should be to automatically deploy a new VCSA and migrate the data from the old to the new and then power down the old. All from the Windows GUI installer.

But it fails to deploy with an unknown error.

If you save and view the installer log it becomes abundantly clear what the failure is. The installer is trying to issue a ‘date’ command at the current VCSA’s command line, and fails because it’s expecting a BASH shell and instead it is getting the default vCenter shell where the BASH shell is disabled.

2017-08-10T07:56:13.446Z - info: VM Identifier for Source VC: 78
2017-08-10T07:56:13.570Z - debug: initiateFileTransferFromGuest error: ServerFaultCode: A general system error occurred: Unknown error
2017-08-10T07:56:13.573Z - debug: Failed to get fileTransferInfo:ServerFaultCode: A general system error occurred: Unknown error
2017-08-10T07:56:13.573Z - debug: Failed to get url of file in guest vm:ServerFaultCode: A general system error occurred: Unknown error
2017-08-10T07:56:13.573Z - error: Error in getting fileData for nodeType. Error: ServerFaultCode: A general system error occurred: Unknown error
2017-08-10T07:56:13.573Z - error: Failed to read the nodetype, Error: A general system error occurred: Unknown error
2017-08-10T07:56:13.574Z - info: Checking if password expired
2017-08-10T07:56:14.994Z - info: Banner from server, 
VMware vCenter Server Appliance 6.0.0.30200

Type: vCenter Server with an embedded Platform Services Controller


2017-08-10T07:56:14.995Z - info: Connection ready
2017-08-10T07:56:15.008Z - info: STDOUT: Last login: Thu Aug 10 07:55:37 UTC 2017 from pc8501.domain.local on pts/0

2017-08-10T07:56:15.008Z - info: STDOUT: Last login: Thu Aug 10 07:56:15 2017 from pc8766.domain.local

2017-08-10T07:56:15.010Z - info: STDOUT: date
exit

2017-08-10T07:56:15.246Z - info: STDOUT: Connected to service

2017-08-10T07:56:15.255Z - info: STDOUT: [?1034h
    * List APIs: "help api list"
    * List Plugins: "help pi list"
    * Enable BASH access: "shell.set --enabled True"
    * Launch BASH: "shell"


2017-08-10T07:56:15.255Z - info: STDOUT: Command> d
2017-08-10T07:56:15.256Z - info: STDOUT: ate

2017-08-10T07:56:15.270Z - info: STDOUT: Unknown command: `date'
Command> exit

2017-08-10T07:56:15.305Z - info: Stream :: close
2017-08-10T07:56:15.306Z - info: Password not expired
2017-08-10T07:56:15.308Z - error: sourcePrecheck: error in getting source Info: ServerFaultCode: A general system error occurred: Unknown error

In order to address the issue you need to change the default shell for the root user. It’s very easy to do, but will also require a password change.

Logon to the current VCSA using ssh as the root user. Enable the BASH shell and start the shell using:

shell.set --enabled True
shell

Now change the root users default shell using

# chsh -s /bin/bash root

Now the installer should at least proceed through that part to deploy the image to the specified server.

References: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2100508

VMware Horizon Client on Debian Stretch — July 17, 2017

VMware Horizon Client on Debian Stretch

In order to install the client on Debian 9 (stretch) I’ve had to get libpng12-0 installed from Jessie here:

https://packages.debian.org/en/jessie/amd64/libpng12-0/download

Then had to create symbolic link for libffi.so.5 to the newer version that’s installed.

$ sudo ln -s /usr/lib/x86_64-linux-gnu/libffi.so.5 /usr/lib/x86_64-linux-gnu/libffi.so.6

 

References: https://communities.vmware.com/thread/545364

How to use vSphere 6.x Certificate Manager — March 10, 2017