I don’t know how this happens but for some reason I end up spending quite a bit of my time trying to get Trend solutions to work. And most of the time it’s in a scenario that hasn’t been covered in the knowledge base articles. I’ve recently been working on a project to create a virtualized test and development environment on Flexpod which involved placing a copy of production behind a firewall. This involves similar IPs and server names but also a problem is that the OfficeScan server requires two vNICs which isn’t really a solution that Trend advise. This problem delayed the project by almost two weeks as I tried numerous fixes and then waited on assistance from support that wasn’t up to scratch to finally getting a Trend employee that really knew his stuff to provide assistance. The configuration I required wasn’t something he had seen in the past but it was definitely something he’d like to see working so we spent a few days trying out different methods to get things working and the steps below is how it was finally fixed.
Requirements:
To install OfficeScan Server on a VM in a DMZ with two vNICs, one with external access to the corporate network and one with internal access via static routes to servers within the DMZ or cell. Only two VMs have ‘egress’ connections to the corporate network and run through an ASA firewall. All other VMs are hidden within the cell in their own domain, do not have internet access and exist across multiple vLANS and also each server VM has multiple vNICs.
Due to the VMs in the cell being a test and development environment for production-based scada systems it has to sit behind a firewall as the scada teams requested that the test and dev environment have the same IP addresses and machine names as their prod environment. Yes, I know that’s crazy and I have brought it up numerous times that this should not be done but I’ve been over-ruled so I’ve just had to deal with it.
Issue:
This is not a recommended configuration from Trend for OfficeScan, they recommend another product for this type of DMZ based protection. We required the OfficeScan server to be able to communicate on two different vNICS, vLANs and IP address on the same ports. This is also not something that Trend has documentation on. In our environment we run a centralised Control Manager that manages the licenses for different OfficeScan servers deployed within the environment. In most instances the different OfficeScan servers sit in different domains. The Control Manager server can only see the newly deployed OfficeScan server on its egress connection, in this case we can say 172.54.21.21. However, all VMs within the cell can only see the OfficeScan server on the internal facing address 172.90.60.99. Trend does have an IPTemplates fix on their knowledge base to allow for multiple vNICs on a client but this doesn’t working in the server side. The issue we faced is that when agents were installed on clients they were being denied on the firewall from reaching the egress connection of 172.54.21.21, and rightly so. Traffic allowed in to the vNIC from the Control Manager server is on port 8080 and the internal client VMs need to communicate with the OfficeScan server on ports <whatever5digitportnum> i.e 19099, and 8080, so allowing internal VMs to hit that egress connection would create a major security risk.
The required fix involved modifying the OfficeScan files so that it could communicate with the Control Manager server for updates and licensing but also allow the client VMs in the cell to communicate on the internal facing IP address/FQDN.
The environment:
The DMZ has been carved off of the Nexus 7Ks within the Flexpod and all traffic from the DMZ has to go to the ASA firewall for routing. From here the ASA firewall rules allow traffic from the DMZ to the Prod DMZ and onto the core switch within the Corporate LAN.
From a VM perspective there are two servers that have ‘egress’ access so that they can communicate between the DMZ network and the Corporate LAN to just specific IP address. The DMZ operates on its own domain which is different to the corporate domain name and no trust exists between domains.
The resolution:
The fix for this issue involved installing OfficeScan server with the IP address of the egress vNIC, 172.54.21.21. Before I began this step I allowed port 8080 on the firewall from the egress vNIC on the Admin server to the Control Manager server in the corporate lan. A quick telnet from the Control Manager server confirmed the change had taken effect.
I won’t go into the details of how to install OfficeScan as this is well covered in the Trend Installation Guide. During the installation enter the IP address of the Control Manager server (as they are on different domains and can’t resolve the name). Once the installation had been completed the next item was to add the new OfficeScan server as a managed server from Control Manager. In Control Manager go to Administration -> Managed Servers and make sure that the new server appears. It should appear with the 172.54.21.21 IP address. This will later be changed to the FQDN but at this point it should just be the IP address.
The first thing to check was that the server was licensed from Control Manager. To do this go to OfficeScan -> Administration -> Product License
This confirms that Control Manager has been configured to manage the updates and the licensing for the new OfficeScan server. In the current configuration it is possible to deploy an agent to the domain controller that also has the egress vNIC. It will register against the 172.54.21.21 address. This is not ideal as the goal is to have all traffic for OfficeScan within the DMZ cell to run on the internal vNICs.
As part of the deployment I had 4 VMs running, the Admin server, domain controller, Server 2008 VM and Windows 7 VM. The test was getting the client VMs, both server2008 and windows 7 communicating with the OfficeScan server as the firewall was blocking these VMs from reaching the egress network on the admin server. These VMs could only see the internal facing IP of 172.90.60.99. One thing to remember here is that all VMs are on the same domain and are set to register their DNS on the internal IP addresses so both the domain controller and the Admin server were registered in DNS as the internal vNIC IPs.
Testing Communications:
The primary problem we had was due to the OfficeScan server having two vNICs even if the agent was installed on the client it could not get its updates from the server. It would always show as trying to connect on the client and would not appear in OfficeScan. What needed to be tested from the client side was if it could connect to the egress link and install the agent, then change the IP address of the OfficeScan server and run an IPXfer from Trend to change it on the client. For more information on IPXfer please check out this KB Article.
- Deploy a test VM with Server 2008 or Windows 7 This VM will have an internal DMZ IP address – 172.90.60.99
- Disable the internal facing vNIC of the OfficeScan server – 172.90.60.99
- Change the VLAN/IP address of the test VM to now have an egress VLAN/IP – 172.54.21.22
- Verify communication with the OfficeScan server on the egress IP address from the client, telnet on port 8080 from client test VM to OfficeScan egress IP – 172.54.21.22.
- Install officescan on test VM using the UNC path as an admin. It will install OfficeScan on the client with the IP address lists as the egress IP of the OfficeScan server.
- Leave if for about 20 to 30 minutes just to make sure
- Enable the internal vNICs on the Admin server
- Change the IP/VLAN for the client VM back to internal VLAN and internal IP address of 172.90.60.100
- Run the following command for ipXfer on the client VM: ipxfer_x64 –s 172.90.60.99 –p 8080 –d domainname
- Run telnet on 8080 to internal IP of OfficeScan server and it connects
On the client VM I checked that the server name/port were now on the internal IP address by checking the Help from the client console
This test basically meant that the egress link was active on port 8080 for communication from the Control Manager and on the internal NIC it was communicating with the internal facing VMs on port 8080.
To fix it on a permanent basis I did the following:
The first step was to check if the client VMs could communicate with OfficeScan is the Master_DomainName IP address was changed to the internal IP address. The Master_DomainName IP address is what is used by clients to reference which server to get it’s updates from. This involved the following steps:
- Verify the following ports have been allowed on the firewall – 8080 from client to server, 5 digit port from server to client e.g 19099
- Go to the installation directory for Trend on the Admin server and edit the ofcscan.ini file. Change the Master_DomainName=DNS-Name. Modifying it to the DNS name will mean that all VMs on the internal network will communicate with the registered DNS IP. Changing this here will also update the Control Manager with the FQDN address rather than an IP. Save the change and go to the OfficeScan console -> Networked Computers -> Global Client Settings -> Click Save. This will update any clients that already have been deployed. Also it will mean that any future VMs will pick up the FQDN rather than IP addresses
- For any VMs that have already been deployed and need to have their agent settings changes from an IP address to the FQDN you can run the IPXfer command: ipxfer_x64 –s <serverFQDN> –p 8080 –d domainname
Now all client installations will use the FQDN and communicate across the internal vNICs for Trend traffic. The OfficeScan server will still get its updates from the Control Manager server.
The last thing to fix are the Web and File Reputation settings as they did not change as part of the above steps. This requires are few different files to be changed. Go to the Trend installation folder on the OfficeScan server.
- Stop ‘OfficeScan Master Service’
- Stop ‘Trend Micro Smart Scan Server’
- In PCCSRVsscfg.ini
Change ‘Local_ScanService=http://<OfficeScanServerName>:8080/tmcss’Change ‘Local_WebScanService=http:// <OfficeScanServerName>:8080’ - In PCCSRVPrivateofcserver.ini
Change the OSCE_URL to reflect the <OfficeScanServerName>.
Then for WSS_URL and WSS_HTTP_URL change the IP address to reflect <OfficeScanServerName>. Then for LWCS_HTTP_URL, change the IP address to reflect <OfficeScanServerName>. - In PCCSRVPrivateSafssnotify.ini
Change ‘Local_ScanService=http:// <OfficeScanServerName>:8080/tmcss’Change ‘Local_WebScanService=http:// <OfficeScanServerName>:8080’Start ‘OfficeScan Master Service’ and ‘Trend Micro Smart Scan Server’.
Next you will need to modify the IPTemplate in the ofcscan.ini file to make sure that clients with dual NIC, not the OfficeScan server but the client, contact the OfficeScan server on the internal IP address
The following KB article can be used for this: http://esupport.trendmicro.com/solution/en-US/1060042.aspx
The main change you will need to make is to add the following to the ofcscan.ini file.
[Global Setting]
#IPTemplate
IPTemplateDeployEnable=1
IPTemplateDeploy0=172.90.60.99
Once that has been added go to the OfficeScan console -> Networked Computers -> Global Client Settings -> Click Save. This will send the change out to the existing clients. On the client go to the registry HKLM->Software->Wow6432Node->Misc and verify the IP address in the IPTemplateDeploy is the internal vNIC IP of the OfficeScan server
And there you have it. How to get OfficeScan to do something it shouldn’t be able to do in the first place.