DSVA Security Update Failed:
Once I had the full Trend Deep Security Manager environment installed I ran the Download Security Updates command to get the latest updates from the Trend website. When trying to update the DSVA I got the following error:
I ran a putty session on the ESX host server (where DSVA security update fails) and saw that there is an entry under vmkernel.log that shows “DSVA not bound”. When I logged into vShield Manager and checked the ESX Host summary and saw that vShield Endpoint was installed but that there were no items listed in Service Virtual Machines. This should show the name of the protected DSVA on that host.
The issue occurs when the DSVA and filter driver improperly bind, causing communication failure between DSVA and the VM to protect. To successfully activate the VM:
- Ensure that the value 169.254.1.1 is bound to Dvfilter-dsa.
- On the vCenter, click the ESXi host.
- Go to Configuration tab > Advanced Settings > Net.
- Make sure that the value of Net.DVFilterBindIPAddress is “169.254.1.1”.
- Make sure that the dvfilter is listening to port 2222.
- On the vCenter, click the ESXi host.
- Go to Configuration tab > Security Profile.
- Under Firewall, click Properties.
- Ensure that the dvfilter is selected and listening to port 2222.
- Restart the filter driver.
- Put the ESXi on maintenance mode. This requires turning off the VMs or migrating them to another ESXi host.
- Connect to the ESXi host via SSH using Putty.
- Run the command “esxcfg-module -u dvfilter-dsa” to unload the filter driver.
- Run the command “esxcfg-module dvfilter-dsa” to reload the filter driver.
- Exit the ESXi from maintenance mode.
- Power on the DSVA.
- On the Deep Security Manager (DSM) console, make sure that the DSVA status is “Managed-Online” and the vShield Endpoint status is “Registered”.
- Activate the VM.
Activation will be successful and the “Dvfilter-dsa: update_sp_binding: DSVA not bound” will no longer appear on the ESXi log.
Deactivating and re-activating the DSVAs fixed this issue.
While working on a recent project for a client utilising Cisco UCS and NetApp for a cloud offering I was tasked with getting Trend Deep Security 9 working for a multi-tenant cloud environment. The primary caveat is that the environment isn’t true end-to-end multi-tenancy as the virtualisation layer is not fully segregated. vCloud Director or another similar tool has not been used but rather the vCloud Suite from VMware and segregation is at the network and storage layers through the use of vDCs on Nexus 7k (network) and SVMs on NetApp Clustered Data OnTap (storage virtual machines). In the production environment Trend Micro professional services were engaged to deliver the original design. Part of the criteria given to them was not to enable multi-tenant mode within the Shared Resources cluster as the tenants would not be managing their own anti-virus protection or scanning. In order to satisfy the requirements of multiple VMware clusters protected by one Anti-virus package a DSM was deployed on each cluster and managed from a central console within the Management cluster. I will go into more of a discussion on the ideal architectural design for multi-tenant anti-virus in another posting.
And so to the beginning of the troubles. No anti-virus solution is really ever straight-forward. There’s a number of policies and exclusions to consider for both operating systems and specific applications and usually there is lack of distinct information within the installation and admin guides. Trend Micro Deep Security Manager is no different. This however is not a huge criticism of Trend, they have to make their documentation as generic as possible for multiple use cases. It does make installation just that bit more frustrating though. You can find the full Deep Security 9.0 Installation Guide here.
Trend Micro Deep Security consists of a number of components that work together to provide protection against viruses and malware in real-time. It can also provide Intrusion Prevention, Web Reputation, Firewalling, File Integrity Monitoring and Log Inspection. It is also available as both agent-based and agentless options. The component of Trend Deep Security are:
- Deep Security Management Console (DSM) – this server (recommended to be virtualised) is the central web-based management console for controlling and managing all Deep Security enforcement components (DSA’s and DSVA’s). The Server is recommended to be Windows Server 2008/2012 R2 64bit.. It is important that it is installed on a different ESXi host to that hosting the VM’s which are protected by the DSM. The DSM should be allocated 8GB and have 4 vCPU allocated. This configuration will be capable of serving up to 10000 agents. The MS SQL database size is relatively small at around 20GB for 10,000 agents.
- Deep Security Relay (DSR) – this server is responsible for contacting Trend Micro’s Security Centre for collection of platform and security updates and relaying this consolidated information back to the DSM and to Agents and Virtual Appliances. The DSR will also be virtualised at Interactive with 8GB ram and 4 vCPU. This configuration will be capable of serving up to 10,000 agents. The Relay has an embedded Agent to provide local protection on the host machine. In the case of multiple relays each will act independently and synchronise their local databases with the Trend Security Centre.
- Deep Security Virtual Appliance (DSVA) – this server is a virtual machine appliance that is installed on every ESXi host. The DSVA enables agentless Deep Security control and management within the hypervisor, providing Anti-Malware, Intrusion Prevention, Integrity Monitoring, Firewall, Web Application Protection and Application Control protection to each VM. The agentless control is only currently available for vSphere 5.1 or earlier. Support for VMware 5.5 will be available in 2014 Q2. The DSVA will communicate directly with the DSM and it is recommended to enable affinity rules within VMware to lock each DSVA to their required ESXi host.
- Deep Security Agent – for non-Windows servers (such as Linux), the agent is deployed directly to the VM’s OS computer, providing Intrusion Prevention, Firewall, Web Application Protection, Application Control, Integrity Monitoring and Log Inspection protection. This is the traditional client-server deployment model and the agent could be included within the imaging process or pushed out from the DSM. DSA will also be necessary on all VM’s within a vSphere 5.5 hypervisor until Q2 2014 (after which a DSVA can be used with vSphere 5.5).
- Smart Protection Server. Web reputation works by clients contacting Trend Micro’s Smart Protection Service on the Internet. Rather than all clients accessing this service, it is possible to deploy Trend Micro’s Smart Protection Server as a VM. The Smart Protection Server will periodically update its URL list allowing it to locally respond to client requests for web reputation ratings. This component is normally part of the Trend Micro Office Scan products and using it may incur an additional licensing fee. Given that the DSVA also caches similar data, this product is not recommended. Hence, the DSVA and DSA will regularly be checking web reputation over the Internet.
- Deep Security Notifier – is aWindows System Tray application that communicates the state of the Deep Security Agent and Deep Security Relay on local computers. A DSA and DSR already contain the Notifier but for Windows guests protected by the DSVA will need ti install the Notifier as a standalone application.
I’ve recently been getting my hands dirty with vShield Manager 5.2. The biggest problem I’ve found with vShield Manager is the wholesale lack of documentation on how the product works and how best to configure it for multiple different environment types. There is general installation and configuration documentation but usually once you go outside of the scope of these there’s a distinct lack of information and it takes a degree in Google Search Dynamics to help find an answer to a problem.
Despite my bitching I did manage to find a solution to a problem for vShield Manager the other day. I’ve been working on bringing a test Flexpod environment up to date. As with a lot of test environments the applications were deployed but never configured. vShield Manager was no different in this case. The error I saw within vShield Manager when I checked the Summary on the ESXi host was: Not applicable to ESX version below 4.1 Patch 3. My immediate question is how can this be possible, I’m running version 5.1 Express Patch 5 and the environment has never had version 4 installed? It turns out this issue is due to the web interface and the steps to resolve the issue can be found here.
My remediation steps for the issue involved:
- Checking the permissions on AD for the service account. It was using the administrator account.
- Created a service account called svc-vsm-<vmname> and gave it domain admin access.
- I then delegated permissions from vCenter datacenter for it.
Once the above was completed I did the following:
- Logged in to the vShield Manager virtual machine using your admin credentials:
default password: default
- To enable root privilege, I ran this command:
- Entered the admin password again.
- Ran this command to configure the terminal:
config t <enter> (config <space> t)
- Ran this command to disable the Web services:
no web-manager <enter>
- Waited for a second or two and then enabled the Web service using this command:
- Ran this command twice to exit the system:
exit <enter> and then exit <enter> again to log out fully.
- Reloaded the client to see the changes and hey presto vShield Endpoint showed up as installed for that ESXi host