OVA Deployment via PowerCLI

Recently i was working on project where i had to deploy RP4VM OVA on more than 400 ESXi hosts. So i decided to write article for  OVA deployment via PowerCLI

Initially i faced a lot of issue like  Deployment was stuck during ‘validation’, i couldn’t run simultaneous deployment in vcenter because of existing vcenter issue etc.

We used naming convention like VxRAIL(DC name)_vRPA1. You can change it if required.

We had to create VMKernel port on esxi hosts for other use, this is also included in last section of the script.

Here is the powercli code which i wrote, explained line by line

1 –  It will load all PowerCLI module if you want to run script from PowerShell

3- Enter the vCenter name and username, password

5 – Please mention path to ova

 7 – Please mention cluster name where you want to deploy OVA

11 – 19 – We used naming convention e.g . VxRail(Clustername)_vRPA1.

You can change as per your need.

23-26 – IP Settings for VMKernel port on ESXi hosts

29-51 – IP setting for OVA

54-81 –  Loop for OVA deployment on ESXi host in cluster

85-99 – Port group and switch info  for VMkernel assignment on ESXi.

Newer version of script available here

& 'C:\Program Files (x86)\VMware\Infrastructure\PowerCLI\Scripts\Initialize-PowerCLIEnvironment.ps1'

Connect-VIServer vcenter -User username -Password password

$filername = "path to OVA"

$storenumber = 'number / name'

#####################################################

$VMNAME = @()

$vm = "VxRail$($storenumber)_vRPA1"
$vm1 = "VxRail$($storenumber)_vRPA2"

$VMNAME += $vm
$VMNAME += $vm1

$i = 0

######################################################3

$subnet = 'subnet'
$subnetprefix = 'x.x.x'
$ipperserver = 1
$firstip = 40
#################################################

## collect IP information #########
$IPS = @()
$vxrailip = Get-Cluster $storenumber | Get-VM | where {$_.name -like "VxRail*"} | Select Name, @{N="IPAddress";E={@($_.guest.IPAddress[0])}}

$split = $vxrailip.IPAddress

$IPbyte = $split.Split(".")
$newucsip = ($IPByte[0]+"."+$IPByte[1]+"."+$IPByte[2])

$startip = "111"
$endip = "112"
$gateway = "126"

$netmask = "subnet"

$ucsipaddress = "$newucsip.$($startip+$_)"
$ucsipaddress1 = "$newucsip.$($endip+$_)"
$ucsgateway = "$newucsip.$($gateway+$_)"

$IPS += $ucsipaddress
$IPS += $ucsipaddress1

$internet = 0

###########################################################3
foreach ($esxhost in (Get-cluster $storenumber | Get-VMHost | sort Name )) {

$portgoup = $esxhost | Get-VirtualPortGroup |where {$_.name -like "Management*"}

$datastore = $esxhost | get-datastore | where {$_.name -like "VxRail*"}


###########################################

$virtual = $VMNAME[$i]
$i ++

$protocol = $IPS[$internet]
$internet ++

#####################################################

$ovfconfig = Get-OvfConfiguration -Ovf $filername

$ovfconfig.Common.ip.Value = $protocol
$ovfconfig.Common.netmask.Value = $netmask
$ovfconfig.Common.gateway.Value = $ucsgateway
$ovfconfig.DeploymentOption.Value = 'Med'
$ovfconfig.IpAssignment.IpProtocol.Value = 'IPv4'
$ovfconfig.NetworkMapping.RecoverPoint_Management_Network.Value = $portgoup
#################################################

$esxhost | Import-VApp -Source $filername -datastore $datastore -OvfConfiguration $ovfconfig -Name $virtual -DiskStorageFormat Thin -Force | Start-VM  -ErrorAction SilentlyContinue


###########################################################
$vswitch = Get-VMHost $esxhost | Get-VDSwitch


$pg = Get-VDswitch -VMHost $esxhost | get-VirtualPortGroup | where {$_.name	-like "vSphere*"}
###############################################################

0..$($ipperserver-1) | %{

$ipaddress = "$subnetprefix.$($firstip + $_)"
$firstip += $ipperserver
New-VMHostNetworkAdapter -VMHost $esxhost -VirtualSwitch $vswitch -IP $ipaddress -SubnetMask $subnet -PortGroup $pg  -ErrorAction SilentlyContinue
}

}
New-DrsRule -KeepTogether:$false -Name vRPAAntiafffinity -Enabled:$true -Cluster $storenumber -VM VxRail$($storenumber)* -Confirm:$false  -ErrorAction SilentlyContinue

Hope this helps… feel free to comment !!

vCenter 6.5 upgrade issue and troubleshooting

This post will give you an overview of vcenter  6.5 upgrade issue and its troubleshooting steps.

Issues encountered during vCenter 6.0 to 6.5 Upgrade

Issue1:- Invalid Appliance (OS) root password

Description: – While Upgrading PSC appliance we need to provide OS *root* password and it was failing with invalid password.

Resolution1: – Reset PSC OS *root* password through web-console and try again.

  • Login to PSC URL <https://PSC_FQDN or IP Address:5480>
  • Go to Administration tab
  • Change Password

Most Important – I have noticed this issue while upgrading or migrating from vCenter 6.5 U3D to 6.5.  For other release i didn’t faced this issue.

Issue2:- VUM Extension blocking Migration

Description: – If VUM Extension is not in use un-register VUM from the vCenter server.

Resolution:- Follow this KB Article https://kb.vmware.com/s/article/1025360

  • Power- Off VUM server 

Issue3:- Unable to detect Network at Step 8

Description: – If vCenter server Management N/W is configured in Distributed Switch and if we provide ESXi host details at step 3 & 4 we are unable to select N/W option at Step8.

Because, “When ESXi is used as the deployment target, non-ephemeral distributed virtual port groups are not supported and hence, not shown drop-down list.

Resolution:- At Step 3 provide – Target Host details & Select Host, At Step4 provide vCenter server details for Appliance deployment.

Note:- If we provide At setep3 & step4 both vCenter details it will fails to migrate data.

vCenter migration no network
vCenter migration no network

Issue4:- AD users unable to access after PSC upgrade, In disjoint domain namespace the domain users might fail to authenticate after you update to vSphere 6.5

Description:- After PSC 6.0 to 6.5 Upgrade AD users unable to access vCenter server.

Resolution: – Delete the AD account with integrated windows authentication and add again

Note:- To perform below activity requires AD rights.

  • Remove PSC’s from AD Object
  • Login to PSC <https://PSC_FQDN or IP Address/PSC>
  • Login with “administrator@vsphere.local”
  • Go to Appliance Settings –> Manage and Rejoin Server to AD Domain.
  • Enterprise AD Domain admin credentials requires from customer.
  • Reboot PSC’s.
  • Validate vCenter authentication with AD credentials.

Issue 5:- Re-Register VUM with vCenter server

Description:- If VUM extension error triggers then follow this procedure to Re-Register VUM with vCenter server.

Resolution: – Follow below steps

  • Login to VUM VM
  • Navigate to C:\Program Files (x86)\VMware\Infrastructure\Update Manager
  • Find ‘ VMwareUpdateManagerUtility.exe’, double click on it
  • Login to the update manager Utility using previously used SSO credentials.
  • Navigate to Re-register to vCenter Server –> enter the SSO credentials and new vCenter IP/FQDN and click Apply. This will re-register VUM to newly deployed VC
  • Once done, close the wizard, and restart the VMware vSphere update Manager Service
  • Login to vCenter and click on plug-ins and enable the VUM plug-in

Issue 6:- Error could not reach the given source vCenter server on “Port 22”

Description: – If SSH is disabled on PSC’s/Appliance vCenter server then this error triggers.

Resolution: – Enable SSH on PSC/Appliance vCenter prior to upgrade.

  • Login to PSC/Appliance vCenter through Web with port 5480
  • Go to Access
  • Click on Edit Option
  • Enable SSH
vCenter SSH login failed
vCenter SSH login failed

Issue 7 – Windows Migration to 6.5 appliance fails 

Description – If source database size is large, then migration pre-check will fail with below error

Resolution – Please check the source database, which table is utilizing more space. Once you find it, either shrink it or compress.  

Note – Please refer VMware KB 51589 here.

vCenter datbase size is full
vCenter datbase size is full

 

Issue 8 – Upgrade of vCenter failed with error, “insufficient space on source export partition “/”

vCenter Source partition error
vCenter Source partition error

Resolution – 

  1. Take a clone or snapshot of source vCenter
  2. Increase the virtual disk (running on low space), follow below VMware KB https://kb.vmware.com/s/article/2145603 
  3. Followed the article to shrink the source vCenter DB                                   service-control –stop –all
    service-control –start vmware-vpostgres
    /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB
    vacuum full;
    vacuum analyze;
    \q
    service-control –start –all
  4. Rebooted the vCenter server appliance and restart upgrad

Issue-9 : vCenter 6.0 migration failing [Error Failed to retrieved remote PSC]

I faced this issue during migration of vCenter 6.0 installed on windows server 2008 R2 to VCSA 6.7u2 with external PSC.

Issue: Error: failed to retrieve version information from remote platform service controller.

vCenter failed to retrieve PSC
vCenter failed to retrieve PSC

Cause: Transport Layer Security (TLS) 1.2 is the default protocol for Platform Service Controller 6.7 by default, while  TLS 1.2 is not supported by default on Windows Server 2008 R2

Resolution – 

Enable TLS 1.2 on Windows Server 2008 R2.
Note: This procedure modifies the Windows registry. Before making any registry modifications, ensure that you have a current and valid backup of the registry and the virtual machine.

  1. Navigate to the registry key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols
  2. Create a new folder and label it TLS 1.2.
  3. Create two new keys with the TLS 1.2 folder, and name the keys Client and Server.
  4. Under the Client key, create two DWORD (32-bit) values, and name them DisabledByDefault and Enabled.
  5. Under the Server key, create two DWORD (32-bit) values, and name them DisabledByDefault and Enabled.
  6. Ensure that the Value field is set to 0 and that the Base is Hexadecimal for DisabledByDefault.
  7. Ensure that the Value field is set to 1 and that the Base is Hexadecimal for Enabled.
  8. Reboot the Windows Server 2008 R2 machine.
  9. Now re-initiate the migration assistance tool it worked perfectly.

Issue 10 – Error: Internal error occurs during vSphere ESX Agent Manager pre-upgrade checks  during Stage 2 of upgrade.

I faced this issue while upgrading vCenter appliance from 6.0 to 6.5

the log shows below error — 

Waiting for VMware ESX Agent Manager.
WARNING: VMware ESX Agent Manager may have failed to start. Continue reading “vCenter 6.5 upgrade issue and troubleshooting”

Reconfigure PSC from embedded to External

Re-configure PSC from Embedded to External

This article will guide you steps to, Reconfigure PSC from embedded to External. To achieve this, first of all you need to install new PSC (it could be windows based or appliance based) in same SSO domain.

Scenario – VMware vCenter 6.0 U3D is setup with embedded PSC.

Step 1 -  Deploy new appliance PSC (Same version / build number of vCenter Server)

Go to Cd-rom –> open vcsa-setup.html and choose install

Provide the required information. You can give vcenter name or ESXi name on which this PSC needs to be deployed.

Select the cluster where PSC needs to be deployed

Give the name of PSC and setup the password

 

Select the existing  SSO Domain name.

Most important – Similarly you can verify SSO site name and domain by below method , if you have multiple SSO site and domains then this steps will help you to cross check correct SSO site.

For VCSA appliance

/usr/lib/vmware-vmafd/bin/vmafd-cli get-domain-name –server-name localhost

/usr/lib/vmware-vmafd/bin/vmafd-cli get-site-name –server-name localhost

Please refer David blog’s for more info SSO Site Name and Domain check

For windows embedded PSC

C:\ProgramData\VMware\vCenterServer\cfg\install-defaults\vmdir.domain-name

“C:\Program Files\VMware\vCenter Server\vmafdd\vmafd-cli.exe” get-ls-location –server-name localhost

Note – Installation path might very based on your environment setup

Please refer William Lam Blogs for More info SSO Site Name and Domain check

Select existing SSO Site name

 

Select the datastore where PSC  needs to be deployed

Enter the appropriate network settings and click finish to start installation.

Step 2 - Once PSC is setup, login to Virtual Center with embedded PSC to verify if new PSC is properly joined to existing SSO domain or not

Please go to the below path

“vcenter installation directory\bin” folder and run below command to see PSC replication partner

vdcrepadmin.exe -f showservers -h localhost -u administrator -w “password”

Note – User should be SSO user.

Here you can see, new PSC is configured as partner with embedded PSC now.

Lets cross check if replication is happening between Embedded PSC and new PSC or not.

 

Step-3 Now we come to the most important steps, to reconfigure PSC from embedded to external

Go to below path

“vcenter installation directory\bin” and run below command

cmsso-util reconfigure –repoint-psc “New PSC FQDN or IP” –username administrator -passwd “password” -domain-name “sso domain name”

Lets verify if new vcenter is pointing to new PSC or not.

Go to advanced setting of vcenter server

Under config.vpxd.sso.sts.uri  — you will see new PSC name

Or go to below path to verify

        VCSA:

         /usr/lib/vmware-vmafd/bin/vmafd-cli get-ls-location –server-name localhost

        Windows VC:

        C:\Program Files\VMware\vCenter Server\vmafdd\vmafd-cli get-ls-location –server-name localhost

       Hope this helps, feel free to comment  🙂

 

vCenter Upgrade Troubleshooting

This article will guide you through various vcenter upgrade troubleshooting steps for windows based vcenter.

  1. You may see below error while upgrading virtual center

2.  To solve this,

a. Ensure tasks mentioned in point 5  Pre-check  are taken care.    Anti-virus / backup software may block VMware DLL / process and                  upgrade will fail.

b. Try the upgrade one more time, if it fails once again then download the “Process explorer” from Microsoft web site

Process Explorer

      c. Check if DLLs mentioned in below KBs are blocked by any process using Process explorer

https://kb.vmware.com/s/article/2149266

                                                                                       

If you find any process using DLL mentioned in above  KB,  kill those process and retry the upgrade.

 3. If it upgrade fails then follow steps 5 to 8 as mentioned in above VMware KB.   (Please ensure snapshot is taken before trying this               step   and all vcenter related services are stopped to delete the files mentioned in steps 5 to 8)

4. You may get different errors, like

Installation of component VCSServiceManager failed with error code ‘1603’. Check the logs for more details

Follow KB- https://kb.vmware.com/s/article/2119768    or  https://kb.vmware.com/s/article/2127284

vCenter upgrade step by step

This article provides vCenter upgrade detailed procedure for windows based virtual center.

Please refer Pre-Check section for validation which needs to be done before upgrade

  1. Upgrade of vcenter / SSO Server (if external)

Login to SSO with service account if applicable (As mentioned in Pre-  check section)

Note – Reboot the server if its not rebooted since long time.

Go to My Computer –> CD Drive –> R.click on Auto run –> Run as administrator

Accept the license agreement

2. Follow the same procedure for virtual center.

Login to virtual center with service account (if applicable) –> go to cd-rom and run as administrator

  

Here you might face some issue, please check Issue

3.   Login to Update manager server via RDP with service account

Select “Server” under vSphere update manager server and follow the required steps

Enter the SSO user password

This does not require reboot, but it would be great if you reboot the server.

Hope this helps, happy reading.  Feel free to leave your comment or suggestions.

vcenter upgrade pre-check

As a professional service team member in DellEMC, i work on upgrade / designing of virtual environment on daily basis for different customers. I am sharing the steps  which i follow while upgrading virtual environment and issues which i faced. This is first post of this series which will guide you to do vcenter upgrade pre-check before upgrade

Part 1 –  Pre-Check or data collection before starting vSphere upgrade

  1. Ensure, we have all the credentials in place like

            a. SSO User password

    b.Service account password (If Virtual Center or VUM Service is running with service account

    c.Data base user password

    d. AD Domain admin credentials (to remove / re-join machine in domain)

      2.  Check the target virtual center hardware and software requirement.

vSphere 6  # for virtual center 6

vSphere 6.5

      3. Check the SQL Server requirement for target virtual center.

DB Compatibility

     4.     Check the compatibility of additional plugin / software integrated with source virtual center to target virtual center

          e.g. To run NSX with vSphere 6.5, it should be above 6.3 or above version.

 

5.  Take backup of SSL certificate. We may need to restore if custom certificates are used

     6. For windows based virtual center, please ensure

   a. Remove anti-virus from server (You can re-install it after upgrade) and if removal is not allowed or possible then please ensure it is          disabled till upgrade is complete.

   b. Remove any backup agent software from Server

   c. If Virtual Center is part of OU in AD with security policy applied, then remove it from OU and move it other OU in AD where security policy are not applied. You can move it back original OU once upgrade is complete.

  d. Reboot the server once above software /settings are removed.

           Note – I would highly  recommend to work on Point 4 because (point a-c) may block VMware service related DLL files and our upgrade                        may fail. This is also recommended by VMware.

 

7.   Login to virtual center via RDP

a. Check the host name and free space on server

 

 Important – Please ensure we have sufficient space for upgrade.  For major upgrade like 5.5 to 6.0 I suggest min 30 GB free should be                               available  and for minor  upgrade 10 GB is fine.

b.  Open service.msc and check the virtual center and its related service status

Note – If virtual center service is configured to run with service account then note down account name. We have to login to virtual                                   center via RDP with this account to start upgrade.

c. Go to Administrative Tool and open ODBC (64 bit)

 

d. Check the advance setting of virtual center to note down SSO server if its embedded or external  (If its external, upgrade the SSO Server first)

 

e. Take backup of Inventory service database # this might come handy if you are upgrading from 5.1 to 5.5 or 5.5 to 6.0

                  On the source virtual center machine, open the command prompt in the vCenter Server and change the directory to                                  vCenter_Server_installation_directory\Infrastructure\Inventory Service\scripts

                  Run the following command at the prompt to back up the Inventory Service database.

                  backup.bat -file backup_file_name

                  When the backup operation finishes, the message Backup completed successfully appears.

 

8.   Login to VUM / Update manager via RDP

   a. Check the free space on server and check host name to ensure we logged in correct server / correct host name is set.

       Note – 10 GB free space is fine for major / minor upgrade

b. Open Service.msc and check VUM service status

Note – If VUM Service is running with service account, note down the name and login with service account to start the upgrade.

 

c. Check ODBC Connection (32 bit)

 

9.   Login to Database server via RDP

     a. Check the disk space on server and host name to ensure we logged in to correct server

      Note – Please ensure we have sufficient free space on server for smooth upgrade process.

    b. Check the MS SQL service status in service.msc

    c. Take backup of virtual center and VUM data base backup before Upgrade

    d. Open SQL Management studio with either

            e.   R.click on vcenter datbase –> Task –> backup

           f. Select Disk under destination

            g. Select the drive where you want to back up the data base. Give the file name “vcenter.bak”  as shown below

e.g. Here I have selected “I” drive and created “RCM Upgrade” folder

         h.  Repeat the same procedure for VUM data base backup

 

Related Article –     Part 2- vCenter upgrade steps

DellEMC Vision

DellEMC Vision (Formally known as VCE Vision)

What is DellEMC Vision Tool

DellEMC Vision software provides health and lifecycle management capabilities embedded in VCE SystemsVision software provides standardized, repeatable processes that enable a system administrator to ensure an optimized, secure, and functional data center or hybrid-cloud environment.

Vision software enables you to know about one or more VCE Systems in a data center, where they are located, and what components they contain. It reports on the health or operating status of all VCE Systems. Use Vision software to rapidly verify that VCE Systems are running the currently supported software or firmware for a particular Release Certification Matrix (RCM). Additionally, you can use Vision software to maintain security hardened infrastructure in accordance with baseline security settings.

Vision software effectively acts as a mediation layer between your VCE System and its existing management tools. The software allows for intelligent discovery by providing a continuous, near real-time perspective of the compute, network, storage, and virtualization resources as a single object, ensuring that the management tools reflect the most current state of the VCE System.

Product Architecture

Each VCE System has a Vision CoreMSM, and an MSP VM deployed and configured as part of the installation process

Data Center Architecture-

Vision software allows you to set up a multisystem configuration within your data center environment.

Vision software provides support for multiple VCE Systems. In a data center environment, an MSM VM can be associated with up to two Vision Core VMs:

Important:Only one MSP VM must be deployed and configured within a cluster. The MSP VM must be associated with the MSM VMs in its local data center for that cluster. If other MSP VMs have been deployed and configured, you must power down these VMs.

Vision software provides support for a clustered environment that includes multiple MSM VMs. These MSM VMs can be configured to run in a single physical data center, or in multiple physical data centers that are geographically separate.

The following illustration shows a single-site environment consisting of three MSM VMs, each of which is associated with a single Vision Core VM. The MSM VMs are configured to form a cluster. The MSM capabilities and functionality are exposed after the deployment and configuration of the MSM VMs. Use the Vision dashboard, the Vision shell, or REST APIs to interact and manage VCE Systems:

In a single-site configuration with one data center, Vision software supports up to three MSM VMs running within the data center. Each MSM VM can be associated with up to two Vision Core VMs.

Vision software also provides support for a multi-site clustering configuration that includes a maximum of three data centers. In a multi-site configuration with three data centers, each data center can have no more than two MSM VMs running, where each MSM VM is associated with up to two Vision Core VMs.

For Complete Instruction and guidelines for clustering, refer to DellEMC Vision Administrator Guide

DellEMC vblock discovery not working in Vision?

    DellEMC Vision, vBlock Discovery failed….??

What is DellEMC Vision,  Well it is a monitoring tool with a capability to check vBlock release compliance matrix (RCM) and Security hardening status for each vblock with baseline security settings. DellEMC Vision periodically discovers Vblock components and one of the ways to check if it is running OK is to check when was the last successful discovery, sometimes vblock discovery does not work or fails.


For Detailed introduction about DellEMC Vision – Please visit  DellEMC Vision


Most common issue we face in or after deploying Vision is, vBlock discovery  fails,   please follow below steps to troubleshoot 

Possible Reason – 1

  1.  Restart of VCE Vision Core didn’t worked.  Login to VCE vision Core VM using putty


          All services are up and running:

1

2

3

4

5

6

7

[root@vision ~]# cd /etc/init.d/

[root@vision init.d]# ./vce-fm-all-services status

vce-fm-master is running (pid=11093 )

vce-fm-adapter is running (pid=11599 )

vce-fm-agent is running (pid=11854 )

vce-fm-net-snmpd (pid=11349 ) is running

vce-fm-naaagent is running (pid=12101 )

    2. Go to logs folder to see if discovery is happening or not

1

2

3

[root@vision init.d]# cd /opt/vce/fm/logs/

[root@vision logs]# grep Performance FMAgent.log |grep "discoverAll and all"

[root@vision logs]#

Nothing in recent logs…

   3. Let’s check previous logs to find the last successful discovery:

[root@vision logs]# grep Performance FMAgent.log* |grep "discoverAll and all"

FMAgent.log.2018-03-16.4: INFO  2018-03-16 09:54:52,574 [main] [update2Listeners] : scheduler.VblocksDiscoveryService - Performance:update2Listeners finished with discoverAll and all(1592ms) totalMem:1053 freeMem:667

FMAgent.log.2018-03-16.4: INFO  2018-03-16 10:04:46,407 [main] [update2Listeners] : scheduler.VblocksDiscoveryService - Performance:update2Listeners finished with discoverAll and all(1216ms) totalMem:1053 freeMem:667

FMAgent.log.2018-03-16.4: INFO  2018-03-16 10:09:49,039 [main] [update2Listeners] : scheduler.VblocksDiscoveryService - Performance:update2Listeners finished with discoverAll and all(1439ms) totalMem:1053 freeMem:648

FMAgent.log.2018-03-16.4: INFO  2018-03-16 10:19:47,603 [main] [update2Listeners] : scheduler.VblocksDiscoveryService - Performance:update2Listeners finished with discoverAll and all(1473ms) totalMem:1053 freeMem:667

FMAgent.log.2018-03-16.4: INFO  2018-03-16 10:24:49,372 [main] [update2Listeners] : scheduler.VblocksDiscoveryService - Performance:update2Listeners finished with discoverAll and all(1245ms) totalMem:1053 freeMem:647

FMAgent.log.2018-03-16.4: INFO  2018-03-16 10:34:39,326 [main] [update2Listeners] : scheduler.VblocksDiscoveryService - Performance:update2Listeners finished with discoverAll and all(1458ms) totalMem:1053 freeMem:666

FMAgent.log.2018-03-16.4: INFO  2018-03-16 10:39:36,794 [main] [update2Listeners] : scheduler.VblocksDiscoveryService - Performance:update2Listeners finished with discoverAll and all(1598ms) totalMem:1053 freeMem:682

[root@vision logs]#

 

Okk.. so last successful discovery was on March 16 2018.

 

Lets dig in dipper for more information

[root@vision logs]# tail -f FMAgent.log

ERROR 2018-03-16 13:43:14,721 [main] [startVblocksDiscoveryService] : scheduler.VblocksDiscoveryService - EULA has not been accepted. discovery/health poll is disabled

ERROR 2018-03-16 13:43:14,721 [main] [startVblocksDiscoveryService] : scheduler.VblocksDiscoveryService - Issue startEulaAcceptance to begin EULA acceptance

INFO  2018-03-16 13:43:14,722 [main] [logEvent] : event.FMEventServiceFactory - publish event: id=EulaEULA has not been accepted. You must accept End User License Agreement to enable discovery/health poll.

 

So, the issue was Eula wasn't accepted. Last time we added new Chassis, then Eula wasn't accepted.

 

Note- Whenever any component added / removed or re-installed from/ in vBlock, then Eula needs to be accepted in VCE Vision Core VM to start discovery

 

 

4. We can follow below steps to accept Eula

 

[root@vision logs]startEulaAcceptance

Starting EULA acceptance: Wed Mar 16 13:50:18 UTC 2018

Enter Name&gt; Mark Strong

Enter Title&gt; Master of the Universe

Enter Licensee Company Name – vipal

Enter Accepting Company Name – vipal

Do you accept the End User License Agreement:[yes/no]- yes

Softcopy of End User License Agreement File created in: /opt/vce/fm/eula/VCE_EULA_V2_2.txt

Wed Mar 16 13:50:58 UTC 2015 EULA has been accepted.


5. VCE Vision Can now discover vBlock


[root@vision logs]# grep Performance FMAgent.log |grep "discoverAll and all"

 

INFO  2018-03-16 15:54:49,895 [main] [update2Listeners] : scheduler.VblocksDiscoveryService - Performance:update2Listeners finished with discoverAll and all(3992ms) totalMem:158 freeMem:95

 

INFO  2018-03-16 15:55:08,445 [main] [update2Listeners] : scheduler.VblocksDiscoveryService - Performance:update2Listeners finished with discoverAll and all(3512ms) totalMem:163 freeMem:115

 

INFO  2018-03-16 16:16:59,372 [main] [update2Listeners] : scheduler.VblocksDiscoveryService - Performance:update2Listeners finished with discoverAll and all(551ms) totalMem:234 freeMem:161

6. Is discovery fails again,


you can run "StopFMAgent" and "StartFMagent" command. Then the vBlock discovery will start as described in point 5.

[root@vision init.d]# stopFMagent

 

Disabling FMagent services from starting automatically...

Waiting for the FMagent services to stop.

vce-fm-master is not running

vce-fm-adapter is not running

vce-fm-agent is not running

vce-fm-net-snmpd is not running

vce-fm-naaagent is not running

[root@vision init.d]# startFMagent

Enabling FMagent services to start automatically…

Waiting for the FMagent services to start.......................................

vce-fm-master is running (pid=10205 )

vce-fm-adapter is running (pid=10724 )

vce-fm-agent is running (pid=10983 )

vce-fm-net-snmpd (pid=10466 ) is running

vce-fm-naaagent is running (pid=11284 )

 

Possible Reason 2

  1. Check NTP is configured on Vision Core server or not. If not, configure it
  2. Check NTP is configured on Vision MSM Server or not, if not please configure it. # You will get option to configure NTP while deploying MSM VM.
  3. Check if Core server is properly added in MSM. If not, run below command in MSM   /opt/vce/multivbmgmt/install/addslibHost.sh
  4. Make Sure, DNS entry is properly configured  for both MSM and Core VM (forward and reverse lookup)

Hope this helps…. happy reading 🙂