Dec 282013
 

I like building applications, and these days that means web applications. The challenge is enjoyable. During my vacation this year I decided to spend some time expanding some code that I’ve been working on over the years. In this case, a tool that runs on my PC that integrates with a database. Previously, I tended towards integrating with a local DB, be it via ODBC, a local text or Excel file, etc. This year I’m setting myself the challenge of integrating with a MySQL database over HTML.

The reason for this is because there is a substantial amount of information in my company’s online database (hosted by NetSuite), but getting access to it for any kind of automation can be tricky. However, it can be done if you’re willing to invest the development time to building an application (both sides of one). But, such a task is not something I’ve ever done before. So I’m combining a hobby and self improvement with the intention of building skills which may be useful at some point in the future at work.

As an example, an application which retrieves system information and helps organize and launch system connections (http, ssh, rdc) is a lot more portable if it retrieves the system information from a central database than something which requires all of the system information to be stored locally. If someone updates the system information and you don’t notice it, it means you could be attempting to connect to the wrong IP or server name. By automating the information retrieval you can save minutes (or over a calendar year, save hours or even days) worth of lookup time.

There are plenty of other uses, such as pulling information from a system, parsing it and then pushing it to a MySQL database directly, and then allowing that information to be displayed in an HTML format. Log parsing, etc., could be streamlined. Even, in one case, parsing the Microsoft hotfixes applied to servers and scrubbing it against a database for which hotfixes have been tested/approved by the manufacturer, would be a great application for my environment. Avaya already has something like that for the CS1000 that was created by the engineers back in the Nortel days. But they don’t have anything like that for the Avaya Aura Contact Center product line, even though they have the audit tool that would be necessary to implement the first half of that endeavor.

While tooling around with the XMLHTTP GET/POST integration for the client tool, I ran into an error on my website that was generated by Mod_Security. “An appropriate representation of the requested resource could not be found on this server. This error was generated by Mod_Security.

Upon further investigation, I managed to capture an error log out of the shared error log on my web host “ModSecurity: Access denied with code 406 (phase 2). Match of “rx ^0$” against “REQUEST_HEADERS:Content-Length” required. [file “/etc/httpd/modsecurity.d/10_asl_rules.conf”] [line “101”] [id “392301”] [rev “5”] [msg “Request Containing Content, but Missing Content-Type header”] [severity “NOTICE”]

The rest of the error is largely environment specific, so I’m omitting that info, but if you’ve run into this error yourself, you know what it looks like.

Here’s what I learned in my search (I’m effectively building a custom browser using Microsoft XMLHTTP— the mechanism isn’t too important, be it Power Shell, vbscript or jscript)

  1. Must declare RequestHeader User-Agent
  2. Must declare RequestHeader Content-Type
  3. For POST, must declare RequestHeader Content-Length

These are not mandatory for all HTML interactions, but some security configurations may require certain headers in order to process an XMLHTTP request. In the case of my web server (shared webhosting) and my custom browser application, for a GET request only the User-Agent and Content-Type were required. However, Mod_Security was configured on the shared webhost to mandate Content-Length.

What triggered this research and error was a typo in my code.

The typo came down to a bad choice in variable declaration. In a foreach ( item in array ) statement, I poorly chose the variables to be foreach ( item in items ) and, I’m sure you can see the typo risk already, I accidentally typed foreach ( item in item ). As such, the foreach loop did not properly iterate over the array… and since the foreach loop set the RequestHeaders, the request headers were not being set. Thus, the mod_security error.

I didn’t catch the typo initially, as the first error message was mostly meaningless and I couldn’t immediately determine the cause. Still, I saw a number of articles (including some wordpress blog support requests for this error, with some fixes involving changing the behavior of Mod_Security). While the Mod_Security error isn’t very meaningful, if you dig into the error_logs deep enough, you’ll find the error message which will lead you to the root cause of the problem. In my case, an HTTP 406 indicating that “Rquest Containing Content, but Missing Content-Type header.”

Once I found the typo in the code and fixed it, the XMLHTTP request worked perfectly (except that mod_security was also configured to require a content-length request header on POST requests, but once I’d fixed the one problem the other was easily to identify and fix.)

Dec 102013
 

If you’ve ever searched for things like

  • Internet Explorer automatically opens and closes
  • How to repair or reinstall Internet Explorer
  • IE opens, flashes then closes immediately

then you know how frustrating it can be to have Windows automatic updates install the latest version of MSIE only to have it not work the next day;

I had this happen to me today and I spent nearly 90 minutes searching for solutions, trying various things solutions and repairing my Windows install.

I attempted:

The System File Checker (run within an Administrative CMD window) turned out to be my solution. A number of %windir%\system32 files were corrupted during the automatic upgrade of MSIE. SFC found and fixed all files (without the need to reboot into safe mode, etc.)

Unfortunately, if I’d been paying attention, I could have prevented this using MSIE >> About.

Nov 122013
 

Issue:

  • Ports, Cards or entire Shelves disable during midnight routine.
  • NWS messages generate during midnight routine
% NWS301 8 0 : -1 -2 -3 -4 -5 -6
%
% NWS101 1 : 24
%
% NWS211 24 : 0 1

Environment:

  • Avaya CS1000, all releases
  • Nortel Meridian-1, all releases
  • Digital phones only

Cause:

  • Cabling issues and/or unplugged phones cause “continuity test” failures during midnight routines.
  • After sufficient number of port-based continuity tests fail a card reports a failure
  • After sufficient number of card-based continuity tests report failures the shelf reports a failure

Solution:

  • If a phone is removed from the jack, restore or de-program
  • If a phone cabling issue exists, fix

Comments:

  • I saw this for the first time when I was working for HellerEhrman. The site’s telecom tech would deploy phones where needed, moving phones from existing workspaces to new workspaces and document in a personal document all unused terminal numbers (TNs) for later re-use but did not de-program them. This was “speedier” for them than removing & reprogramming TNs. Doing this allowed them to save the time of programming the entire TN, they just plugged a new phone in, re-enabled the port, changed the DN and they were good.
  • However, users began reporting phones were disabling during midnight routine and had to be manually re-enabled next business morning.
  • Issue escalated to me (Firm-wide Telecom team).
  • I’d never seen this particular issue before and did not know root cause.
  • I performed routine troubleshooting and recommended several corrective actions, including routine maintenance (cleaning up TNs, etc.) but having no authority over the site tech (not being able to force them to do the recommended work and I did not know that the absence of routine maintenance was the proximate cause) I was told to escalate to Nortel (via our Service Provider).
  • Service provider had not seen it before and escalated to Nortel
  • Nortel indicated performance of routine maintenance. i.e., clean up all programmed TNs that were not going to be put back into service or reconnect a phone to any TN that needed to remain.
  • Issue resolved.

I’ve seen a couple of these tickets recently at my place of employment. So far each one appears to be the same cause/solution. I’ll post a comment later if I learn anything new.

Nov 072013
 

Issue

When IP Phones enter a reboot loop, attempt to “upgrade”, fail, then reboot again, or
When IP Phones enter a reboot loop, attempt to “upgrade”, fail with “FW authentication failure”, then reboot again

Environment

Avaya CS1000

UNIStim 5.0 or earlier

Avaya IP Phone 1100, Avaya IP Phone 1200

Cause

UNIStim firmware is digitally signed.

Signature has an expiration date.

UNIStim versions prior to 5.0 had shorter expiration dates.

New IP Phone hardware will not load firmware with expired signatures.

Source: http://downloads.avaya.com/css/P8/documents/100152833

Solution

Use UNIStim 5.1 or later firmware.

Avaya has applied a digital signature with a 10 year expiration date to UNIStim 5.1 and later.

UNIStim 5.5.1 (C8T) released in Aug 2013.

I updated my Google drive table of UNIStim firmware releases.

 

Oct 182013
 

In a previous post, I talked about an experience I had in documenting for co-workers how to set up the CS1000E. The root cause of that documentation was the excessive amount of time I spent cleaning up after a field person who refused to install this properly and the subsequent complaints from customers on why the phone system went down during maintenance windows.

Having the ability to add system redundancy (or resiliency) does not necessarily mean that a customer requires said redundancy, but sometimes the lack of the redundancy is not a factor in the customer’s thinking. Sometimes, knowing you can prevent something (if you pay the associated costs) is not worth the money or time.

This is a different decision from arguing that something doesn’t work a particular way– and today I ran into this problem with a customer who did not have the necessary redundant network connections and experienced an outage as a result.

In this particular case, either the cable and/or data switch port went bad. Had the customer installed the necessary redundancy, the failure of a single port would not have been noticed and the system would have kept on trucking.

As part of the post event discussion, I walked them through how the architecture supported additional redundancy and the extent to which that redundancy can be expanded. I decided to work up a diagram to more fully explain what I was talking about.

This diagram shows a CPPM CS (Call Processor Pentium Mobile – Call Server) connected to the passthrough port on the MGC. The passthrough port permits you to simulate increased CS redundancy to the ELAN network by passing though to either active MGC ELAN interface.

The downside of this connection is that if the MGC undergoes maintenance, or the cable goes bad, you still have a single-point-of-failure.

I would do this primarily only when the environment is also going to deploy redundant TLAN/ELAN data switches for increased network resiliency. Otherwise, connecting the CS directly to the ELAN network makes more architectural sense to me. (That way if you’re installing loadware on the MGC associated with the CS, you don’t cause outages to the entire system when the MGC is rebooted– although there are architectural decisions that can be made to work around some of that as well but we’re not going to cover every possible scenario in this article. Please feel free to comment below to engage in a discussion if you have questions or want to share your observations.)

The diagram also shows the redundant connections from the MGC (faceplate & rear-chassis) connected to a redundant data network. NOTE: I do not show the data switch connectivity with the rest of the network. That’s sort of beyond the scope of the CS1000 resiliency feature. You can, I’m sure, get the gist of it from this article.

Sep 262013
 

A co-worker was recently tasked with providing cross training for a product which I do not have much experience with on the topic of T1 troubleshooting and alarm clearing. After getting this class, I decided it would be fun to put together something similar (in blog format) for CS1000 T1 alarm clearing.

Receiving an alarm

There are a variety of PRI alarms, but we’ll take one of them as an example:

DTA021 [loop]

Interpreting the alarm

Some systems have part of the alarm lookup database on-system which can be accessed via the Overlay Loader (OVL000 and a > prompt) using the ERR command

>ERR DTA021

If the alarm library is loaded with that alarm, then you’ll get the help text. If not, you’ll get an error:

OVL441 Help text not found for error code: [code]

All alarms in the documentation are in 4 digit length after the 3 letter alarm group code. DTA are digital trunking alarm. 021 is the specific alarm. Finding it in the documentation can be done by searching for DTA0021. From the documentation we get the alarm text:

Frame alignment alarm persisted for 3 seconds

Responding to the alarm

Let’s talk briefly about some of the different tools available for troubleshooting:

  • LD 60 / Digital Trunk Interface and Primary Rate Interface Diagnostics
  • LD 96 / D-channel Diagnostics
  • LD 20 / Print Routine 1 – Use to identify a trunk’s association with a Route Datablock (RDB).
  • LD 21 / Print Routine 2 – Used to list trunk members to determine trunk group associations between PRIs/DTIs.
  • LD 22 / Print Routine 3 – Used to print common equipment (CEQU) and D-channel configuration (ADAN DCH)
  • LD 73 / Digital Trunk Interface – Used to check clocking (DDB)

Digital Trunk Interface and Primary Rate Interface Diagnostics

DTI and PRI diagnostics (LD 60) cover a variety of tasks, you can: enable/disable loops, clocking, individual bearer channels (B-channels or BCH) and print/clear counters.  For a full list of commands, see the Software Input/Output Reference – Maintenance (NN43001-711).

Commands:

STAT
STAT [loop]
STAT [loop channel]
Show status of all loops or loop specified. Loop status include loop state and BCH state.
LCNT
LCNT [loop]
List counters
RCNT
RCNT [loop]
Reset counters
SSCK [core]
SSCK [loop shelf]
Show system clock. Includes which circuit is being used for primary clocking and clock state.
SWCK Swap clock from current active to current standby
TRCK [Source] Set clock controller tracking to PCK/Primary Clock, SCK/Secondary Clock, FRUN/Free run-no clock.
ENLL [loop] Enable loop
ENCH [loop channel] Enable B-channel
DISL [loop] Disable loop
DISI [loop] Disable loop when idle. Disables any IDLE channel then waits till other channels are disabled. Loops until all B-channels are disabled then disables loop.
DSCH [loop channel] Disable B-channel

D-channel Diagnostics

DCH Diagnostics covers: enable/disable d-channels, d-channel monitors, and work with MSDL or TMDI cards. On larger legacy 1000M systems, the Multipurpose Serial Data Link (MSDL) card is used to provide D-channel functionality. On smaller 1000M systems and newer 1000E systems, the D-channel functionality is built into the TMDI (T1 Multipurpose Digital Interface) card. In this article, we will not be discussing troubleshooting D-channel diagnostics for MSDL cards on larger 1000M systems.

Commands:

STAT DCH
STAT DCH [dch]
Show status of all/specific DCH.
ENL DCH [dch] Enable DCH
DIS DCH [dch] Disable DCH
STAT TMDI [card] Show status of TMDI. (CS1000M small system)
STAT TMDI [loop shelf card] Show status of TMDI. (CS1000E)
DIS TMDI [card] Disable TMDI. (CS1000M small system)
DIS TMDI [loop shelf card] Disable TMDI. (CS1000E)
SLFT TMDI [card] Selftest TMDI. (CS1000M small system) Performs multiple hardware tests to verify TMDI is functional.
SLFT TMDI [loop shelf card] Selftest TMDI. (CS1000E) Performs multiple hardware tests to verify TMDI is functional.
ENL TMDI [card [fdl]] Enable TMDI. (CS1000M small system) Optional FDL/Full Download of TMDI EPROM.
ENL TMDI [loop shelf card [fdl]] Enable TMDI. (CS1000E) Optional FDL/Full Download of TMDI EPROM.
PLOG DCH [dch]

Print Routine 1

The command architecture for the CS1000 is built on the older Meridian-1 systems, which in turn is built upon the even older SL-1 systems. When the SL-1 hardware architecture was replaced or improved, Nortel introduced new commands or Overlays as needed, all while keeping the essential command structure introduced with the first SL-1 system in the mid-1970s.

Print Routine 1 covers peripheral programming, including the bearer channel (BCH) configuration for a T1. From the Terminal Number configuration of a BCH, it is possible to identify the route membership for a particular channel, and by extension the T1. (While it is technically possible to configure different channels within a T1 to belong to multiple routes, I’ve never seen this and excepting MUXed circuits I am not aware of any reason why it might be done.)

Print Routine 2

Print Routine 2 covers customer datablock configurations, including route datablock (RDB) settings. By using the List Trunk Members command, it is possible to identify all of the BCH (and by extension all the T1s) that belong to a particular route (i.e., trunk group).

Print Routine 3

Print Routine 3 covers hardware and system configuration data, such as the Common Equipment (CEQU) datablock and Action Device and Number (ADAN) datablock, the latter of which is used to store information about D-channel configuration.

Digital Trunk Interface

When building a PRI/DTI in a CS1000 system for the first time, the Clock Controller and Alarm Threshold values must be set. For systems in the USA, the DDB (digital data block) configuration record contains the relevant configuration settings.

Diagnostics vs Print Routines & configuration

LD 60 and 96 are used primarily for diagnostics. Overlays 20-22 and 73 would be used to configuration review to assist with diagnostics. For the purposes of this article, we will assume that a configuration issue is not at fault. Perhaps in some future article I might cover PRI configuration in more detail.

References

NN43001-611 Software Input/Output Reference – Administration

NN43001-711 Software Input/Output Reference – Maintenance

NN43001-712 Software Input/Output Reference – System Messages

NN43001-301 ISDN Primary Rate Interface Installation and Commissioning

Sep 192013
 

Tagged & Unregistered Frame Diagram

Recently found myself troubleshooting Untagged and Unregistered frame filtering on an Avaya Ethernet Routing Switch. This is a quick tutorial for future discussions with other engineers about how filtering works on an Avaya Ethernet Switch.

A quick description of tagged vs registered:

  • A tagged frame is a frame which has an 802.1q VLAN ID tag on the packet.
  • A registered frame is a frame which has an 802.1q VLAN ID tag on the packet that matches the VLAN MEMBERS configuration on the port.

In the above diagram, if the packet egressing from the first data switch is tagged with VLAN 10 (PVID or Primary VLAN ID 10), then the packet is both tagged and registered when it ingresses on the second data switch. However, if the packet is tagged with VLAN 20, the packet is tagged but unregistered when it ingresses on the second data switch.

Use of untagged filtering or unregistered filtering is for environments where the administrator wishes to protect against mistakes in recabling network devices or vlan configuration mistakes. One of the historical issues that has happened in the past with Nortel Ethernet Routing Switches is that if an unregistered frame is received and the receiving data switch does not know what to do with it, it may divert the packet to VLAN 1 (the default VLAN ID on all data switches). This can result in accidental broadcasts of extraneous packets.

A few examples of network changes that might result in problems:

  • Accidentally connecting a server to the second data switch trunk port– the server packets are filtered by filter-untagged-frame enable.
  • Accidentally connecting a different data switch to the second data switch trunk port– if the vlan membership and PVID settings are different, the server packets can be filtered by filter-untagged-frame or filter-unregistered frame.
  • The first data switch is factory reset or the trunk port is configured with VLAN membership that need not be extended to the second data switch– if the 802.1q VLAN ID on the packet is not one of the VLAN member IDs on the ingress port, then the packet will be filtered by filter-unregistered-frame. If the first data switch port is configured as a non-trunk port (any other tagging configuration: tagging untagpvidonly, tagging untagall, tagging tagpvidonly) and the packet is untagged and sent to the second data switch, then the filter-untagged-frame setting may cause the packet to be filtered on ingress.

Michael McNamara posted an article back in 2007 discussing an issue with Avaya IP Phones where filter-unregistered-frame enable caused problems with IP Phone registration. The relevant excerpt is as follows:

The option (vlan ports 1-46 filter-unregistered-frames disable) was added after an issue was discovered when trying to upgrade the firmware on the IP phones. The filter-unregistered-frames is enabled by default and should be disabled to avoid and issues with upgrading the firmware on the IP phones. We are attempting to investigate further with Nortel and our voice vendor…

Take away:

  • The way I read the documentation, and I haven’t had a lot of opportunity to thoroughly test this, filtering is ingress only. This means you’re consuming bandwidth with improper egress port configuration. Depending on your environment, this wasted bandwidth might be minor or major.
  • Use of VLAN ID is strongly frowned upon in a multiple VLAN environment. Don’t assign PVID 1 or VLAN ID to any ports (i.e., remove VLAN 1 from all ports, because VLAN ID 1 is on everything in factory default configurations.) If you’re not doing any VLANing or any Layer 3 routing (i.e., simply Layer 2 switching only), then VLAN ID 1 is fine– but as soon as you start tagging packets on that switch you should stop using VLAN ID 1.
  • Filtering is best used on trunk ports to prevent broadcast storms or extraneous packets from being broadcast into a data switch (and the impact of extraneous packets from diverted unregistered frames is minimized if you do not use VLAN ID 1.)

What to do with this information:

  • Review all ports for VLAN membership or PVID of 1– if found and if you are a multi-VLAN environment, convert VLAN ID 1 to another VLAN ID on all ports then remove VLAN ID 1 from all ports (including PVID 1).
  • Review all ports set with tagging tagAll (including multi-link trunk (MLT) ports)
    • Compare VLAN membership and PVID settings on each side of a trunk connection.
    • Make sure VLAN membership matches on both sides.
    • Make sure the PVID ID on each side is a VLAN member on the opposite side.
    • Make sure the PVIDs match on both sides (I can’t think of any valid design reasons why PVIDs might not match.)
  • Review all trunk ports (tagging tagAll) and set filter-untagged-frame enable
  • Review all trunk ports (tagging tagAll) and set filter-unregistered-frame enable
  • Review all non-trunk ports (access ports) and set filter-untagged-frame and filter-unregistered-frame based on the network design
    • Note: If the port will not be receiving 802.1q tagged packets on ingress, then filter-untagged-frame and filter-unregistered-frame should be disabled.

Aug 022013
 

This morning was an interesting chain of events:

  • Contacted by a coworker who was wanting to use one of the parser tools I’ve put up (http://www.datarave.net/meridian/dch.php), but said my website was down.
  • Investigated and confirmed.
  • Attempted to access the host provider (hostmonster.com), website did no resolve DNS, or ping, or return a webpage.
  • Looked up phone number from old email (hosting plan renewal notification). Called. Line connects to silence. After extended silence, the line rings 4 times, then goes silent again. After a bit more silence the call is disconnected. Retested several times, always the same results.
  • Googled
  • Found http://iidrn.com/hostmonster.com.html said that the site is really down. Comments on site indicate that this is one of many outages this last week.
  • Checked twitter https://twitter.com/search?q=hostmonster&src=typd&mode=realtime found multiple people complaining about hostmonster and affiliate outages
  • Checked @hostmonster and @hostmonsterhelp, no updates

Site is up right now, but I expect it to not be stable.

Apr 032013
 

  • LD 48
  • Enabling Application Module Link (AML) traces
    • enxp <msgi/msgo> <aml_id> <exclude_priority> … <exclude_priority>
      Example:
      enxp msgo 16 1
      enxp msgi 16 1
    • enl <msgi/msgo> <aml_id>
      Example:
      enl msgi 16
      enl msgo 16
  • Disabling
    • dis <msgi/msgo> <aml_id>
    • dsxp <msgi/msgo> <aml_id>

 

Apr 012013
 

For a while I’ve been having a problem with Google Chrome. One of the recent updates disabled and prevented Norton Identify Protection from running within Chrome. The biggest problem here is that I’ve taken to using http://grc.com/passwords to generate 12-24 length, completely random passwords… and then storing it in Norton Internet Protection. So if I want to browse any place that I have an account (using Chrome), I now have to load MSIE, open the Password Vault, then find the password (manually search), then pull the password & username and populate it in the Chrome browser window.

Let me tell you, that was highly annoying.

So I swapped back to MSIE until I figured out the solution.

Solution #1

  1. Norton Toolbar will be loaded only when you visit/navigate/open any webpage e.g.. Browse to www.google.com/gmail.com and check toolbar appears with little green icon at right top corner or not.
  2. Type ” chrome://extensions/ ” in address bar and check Norton Identity Protection “enabled” with tickmark, If it was disabled click on check box with tick mark
  3. If you still face the problem, open chrome://extensions/ click on remove for Norton Identity Protection
  4. Follow below steps to add Norton Identity Protection plugin into chrome

If that does work (and it didn’t for me), Solution #2

  1. Open N360 V6 engine folder ( e.g.C:\Program Files\Norton360\Engine\) X is the product version.
  2. Go to “exts” folder and copy the chrome.crx file to any other location
  3. Open that file with WinRAR or 7Zip and extract all the contents to any folder.
  4. Open Chrome and type chrome://settings/extensions in address bar
  5. Select “Developer mode” using check mark and click on “load unpacked extension”
  6. Browse to the folder where you have extracted all the contents and Click OK
  7. Now you will have Norton identity protection in chrome.

Then I had the problem where Chrome kept telling me that the manifest_version wasn’t set to 2 in the manifest.json. So I went and opened the manifest.json and, sure enough, there was no manifest_version key within the json file. Another Google Search and voila— Simple as adding manifest_version : 2, to the manifest.json file of the Norton Internet Security Chrome Extension.

Sources:

Update

This process did not fix the issue I was having. It turns out that the root cause of the problem was that Chrome would not update the Norton Internet Protection extension. It was still using the 2012 version and even after upgrading to the 2013 extension it wouldn’t remember it between shutdown-restarts of Chrome (let alone the OS).

I ended up engage Norton Internet Security support. The level 1 support tech spent 3 hours doing the following (at no time did I sign into Chrome with my google profile to force synchronization, not that this worked anyway):

  • Removing the Norton Identity Protection (NIP) 2012 extension, closing Chrome, restarting Chrome. NIP 2012 extension still installed.
  • Removing NIP 2012 extension, installing the NIP 2013 extension (dragging the chrome.crx from the %programfiles(x86)%\norton internet security\engine\(version)\exts folder into the extensions window in Chrome), verifying that NIP 2013 was installed (but not working), closing Chrome, restarting Chrome. NIP 2012 extension still installed.
  • Uninstalling Google Chrome via Add/Remove Programs (start > run > appwiz.cpl). Reinstalling Chrome via google.com.
    • And here is where something magical happened in the troubleshooting process– I noticed that not only did Chrome (post re-install) still have NIP 2012 extension installed, but it also had every other Chrome extension installed… and I hadn’t signed into Chrome with my Google profile… so no extensions should have been installed.
  • Repeating the above several times expecting different results than were obtained the first time.
  • Ending the call with a promise that the level 2 support tech would call back within 48 hours.

I had planned to google (do you see the funny?) how to purge the Chrome profile from my PC and then try relaunching Chrome, but I hadn’t gotten around to it before the level 2 support tech called. (I’m getting along fine in MSIE, just like I have for years.) The level 2 support tech called this morning and within 45 minutes we’d found the solution for my problem:

  • Google location of Chrome profile [Superuser article] (using MSIE)
  • Close Google Chrome
  • Browse to %localappdata%\google\chrome\user data\
  • Zip all files in profile
  • Delete all files (except the backup zip)
  • Launch Google Chrome
  • Verify NIP 2012 not installed.
  • Install NIP 2013 extension (see above)
  • Close & reopen Chrome to verify problem fixed

I have to set up Chrome again (mostly) from scratch, but Google helps by pulling down my Google profile to the Chrome installation (bookmarks, extensions, etc.) which saves me a ton of time.