I don’t know how this happens but for some reason I end up spending quite a bit of my time trying to get Trend solutions to work. And most of the time it’s in a scenario that hasn’t been covered in the knowledge base articles. I’ve recently been working on a project to create a virtualized test and development environment on Flexpod which involved placing a copy of production behind a firewall. This involves similar IPs and server names but also a problem is that the OfficeScan server requires two vNICs which isn’t really a solution that Trend advise. This problem delayed the project by almost two weeks as I tried numerous fixes and then waited on assistance from support that wasn’t up to scratch to finally getting a Trend employee that really knew his stuff to provide assistance. The configuration I required wasn’t something he had seen in the past but it was definitely something he’d like to see working so we spent a few days trying out different methods to get things working and the steps below is how it was finally fixed.
To install OfficeScan Server on a VM in a DMZ with two vNICs, one with external access to the corporate network and one with internal access via static routes to servers within the DMZ or cell. Only two VMs have ‘egress’ connections to the corporate network and run through an ASA firewall. All other VMs are hidden within the cell in their own domain, do not have internet access and exist across multiple vLANS and also each server VM has multiple vNICs.
Due to the VMs in the cell being a test and development environment for production-based scada systems it has to sit behind a firewall as the scada teams requested that the test and dev environment have the same IP addresses and machine names as their prod environment. Yes, I know that’s crazy and I have brought it up numerous times that this should not be done but I’ve been over-ruled so I’ve just had to deal with it.
This is not a recommended configuration from Trend for OfficeScan, they recommend another product for this type of DMZ based protection. We required the OfficeScan server to be able to communicate on two different vNICS, vLANs and IP address on the same ports. This is also not something that Trend has documentation on. In our environment we run a centralised Control Manager that manages the licenses for different OfficeScan servers deployed within the environment. In most instances the different OfficeScan servers sit in different domains. The Control Manager server can only see the newly deployed OfficeScan server on its egress connection, in this case we can say 18.104.22.168. However, all VMs within the cell can only see the OfficeScan server on the internal facing address 22.214.171.124. Trend does have an IPTemplates fix on their knowledge base to allow for multiple vNICs on a client but this doesn’t working in the server side. The issue we faced is that when agents were installed on clients they were being denied on the firewall from reaching the egress connection of 126.96.36.199, and rightly so. Traffic allowed in to the vNIC from the Control Manager server is on port 8080 and the internal client VMs need to communicate with the OfficeScan server on ports <whatever5digitportnum> i.e 19099, and 8080, so allowing internal VMs to hit that egress connection would create a major security risk.
The required fix involved modifying the OfficeScan files so that it could communicate with the Control Manager server for updates and licensing but also allow the client VMs in the cell to communicate on the internal facing IP address/FQDN.
DSVA Security Update Failed:
Once I had the full Trend Deep Security Manager environment installed I ran the Download Security Updates command to get the latest updates from the Trend website. When trying to update the DSVA I got the following error:
I ran a putty session on the ESX host server (where DSVA security update fails) and saw that there is an entry under vmkernel.log that shows “DSVA not bound”. When I logged into vShield Manager and checked the ESX Host summary and saw that vShield Endpoint was installed but that there were no items listed in Service Virtual Machines. This should show the name of the protected DSVA on that host.
The issue occurs when the DSVA and filter driver improperly bind, causing communication failure between DSVA and the VM to protect. To successfully activate the VM:
- Ensure that the value 169.254.1.1 is bound to Dvfilter-dsa.
- On the vCenter, click the ESXi host.
- Go to Configuration tab > Advanced Settings > Net.
- Make sure that the value of Net.DVFilterBindIPAddress is “169.254.1.1”.
- Make sure that the dvfilter is listening to port 2222.
- On the vCenter, click the ESXi host.
- Go to Configuration tab > Security Profile.
- Under Firewall, click Properties.
- Ensure that the dvfilter is selected and listening to port 2222.
- Restart the filter driver.
- Put the ESXi on maintenance mode. This requires turning off the VMs or migrating them to another ESXi host.
- Connect to the ESXi host via SSH using Putty.
- Run the command “esxcfg-module -u dvfilter-dsa” to unload the filter driver.
- Run the command “esxcfg-module dvfilter-dsa” to reload the filter driver.
- Exit the ESXi from maintenance mode.
- Power on the DSVA.
- On the Deep Security Manager (DSM) console, make sure that the DSVA status is “Managed-Online” and the vShield Endpoint status is “Registered”.
- Activate the VM.
Activation will be successful and the “Dvfilter-dsa: update_sp_binding: DSVA not bound” will no longer appear on the ESXi log.
Deactivating and re-activating the DSVAs fixed this issue.
While working on a recent project for a client utilising Cisco UCS and NetApp for a cloud offering I was tasked with getting Trend Deep Security 9 working for a multi-tenant cloud environment. The primary caveat is that the environment isn’t true end-to-end multi-tenancy as the virtualisation layer is not fully segregated. vCloud Director or another similar tool has not been used but rather the vCloud Suite from VMware and segregation is at the network and storage layers through the use of vDCs on Nexus 7k (network) and SVMs on NetApp Clustered Data OnTap (storage virtual machines). In the production environment Trend Micro professional services were engaged to deliver the original design. Part of the criteria given to them was not to enable multi-tenant mode within the Shared Resources cluster as the tenants would not be managing their own anti-virus protection or scanning. In order to satisfy the requirements of multiple VMware clusters protected by one Anti-virus package a DSM was deployed on each cluster and managed from a central console within the Management cluster. I will go into more of a discussion on the ideal architectural design for multi-tenant anti-virus in another posting.
And so to the beginning of the troubles. No anti-virus solution is really ever straight-forward. There’s a number of policies and exclusions to consider for both operating systems and specific applications and usually there is lack of distinct information within the installation and admin guides. Trend Micro Deep Security Manager is no different. This however is not a huge criticism of Trend, they have to make their documentation as generic as possible for multiple use cases. It does make installation just that bit more frustrating though. You can find the full Deep Security 9.0 Installation Guide here.
Trend Micro Deep Security consists of a number of components that work together to provide protection against viruses and malware in real-time. It can also provide Intrusion Prevention, Web Reputation, Firewalling, File Integrity Monitoring and Log Inspection. It is also available as both agent-based and agentless options. The component of Trend Deep Security are:
- Deep Security Management Console (DSM) – this server (recommended to be virtualised) is the central web-based management console for controlling and managing all Deep Security enforcement components (DSA’s and DSVA’s). The Server is recommended to be Windows Server 2008/2012 R2 64bit.. It is important that it is installed on a different ESXi host to that hosting the VM’s which are protected by the DSM. The DSM should be allocated 8GB and have 4 vCPU allocated. This configuration will be capable of serving up to 10000 agents. The MS SQL database size is relatively small at around 20GB for 10,000 agents.
- Deep Security Relay (DSR) – this server is responsible for contacting Trend Micro’s Security Centre for collection of platform and security updates and relaying this consolidated information back to the DSM and to Agents and Virtual Appliances. The DSR will also be virtualised at Interactive with 8GB ram and 4 vCPU. This configuration will be capable of serving up to 10,000 agents. The Relay has an embedded Agent to provide local protection on the host machine. In the case of multiple relays each will act independently and synchronise their local databases with the Trend Security Centre.
- Deep Security Virtual Appliance (DSVA) – this server is a virtual machine appliance that is installed on every ESXi host. The DSVA enables agentless Deep Security control and management within the hypervisor, providing Anti-Malware, Intrusion Prevention, Integrity Monitoring, Firewall, Web Application Protection and Application Control protection to each VM. The agentless control is only currently available for vSphere 5.1 or earlier. Support for VMware 5.5 will be available in 2014 Q2. The DSVA will communicate directly with the DSM and it is recommended to enable affinity rules within VMware to lock each DSVA to their required ESXi host.
- Deep Security Agent – for non-Windows servers (such as Linux), the agent is deployed directly to the VM’s OS computer, providing Intrusion Prevention, Firewall, Web Application Protection, Application Control, Integrity Monitoring and Log Inspection protection. This is the traditional client-server deployment model and the agent could be included within the imaging process or pushed out from the DSM. DSA will also be necessary on all VM’s within a vSphere 5.5 hypervisor until Q2 2014 (after which a DSVA can be used with vSphere 5.5).
- Smart Protection Server. Web reputation works by clients contacting Trend Micro’s Smart Protection Service on the Internet. Rather than all clients accessing this service, it is possible to deploy Trend Micro’s Smart Protection Server as a VM. The Smart Protection Server will periodically update its URL list allowing it to locally respond to client requests for web reputation ratings. This component is normally part of the Trend Micro Office Scan products and using it may incur an additional licensing fee. Given that the DSVA also caches similar data, this product is not recommended. Hence, the DSVA and DSA will regularly be checking web reputation over the Internet.
- Deep Security Notifier – is aWindows System Tray application that communicates the state of the Deep Security Agent and Deep Security Relay on local computers. A DSA and DSR already contain the Notifier but for Windows guests protected by the DSVA will need ti install the Notifier as a standalone application.