Outbound mail server update

Earlier this year, we identified a spam filtering problem on AT&T’s end causing mail to be rejected when sending to pacbell.net and prodigy.net e-mail addresses (and only to those addresses) from our NCCN.net mail server.

Extensive, multi-month efforts to resolve the problem via AT&T and third-party correspondence were unsuccessful, so we have deployed a new outbound e-mail server.  After testing to make certain that it is not blocked at AT&T’s end (nor elsewhere), we have transitioned to sending all NCCN outbound e-mail through the new server.  This has fixed the spam filtering issue, and also adds additional capacity and redundancy to our email services.

No changes should be necessary to your e-mail settings.  If you are now having trouble sending outbound e-mail for your NCCN.net address, verify that your outbound mail server is smtp.nccn.net.  You can also call us at (530) 478-9822, Ext. 1, to troubleshoot connection problems.

Posted in service update | Comments Off

NCCN.net mail delivery delays

On Tuesday evening, April 28, the server handling mail for NCCN.net developed network issues that prevented new inbound messages from reaching their mailboxes.  Customer access to those mailboxes was not affected, so customers continued to see their existing e-mail but were not receiving anything new.

The problem has been resolved as of 10 AM Wednesday, April 29.  Inbound messages received during that time period were queued in our spam filtering server, and that backlog is being processed for delivery.

No messages should be lost, but it may take several hours to finish processing the backlog. If you were expecting e-mail to arrive during the affected hours and it has not been delivered by 3 PM Wednesday, April 29, give our office a call at (530) 478-9822, Ext. 1 for technical support.

Posted in unplanned outage | Comments Off

Web hosting server down

As of noon Monday, April 27, our secondary web hosting server (running about 50 websites) is experiencing problems.  Both email and web access are down.  The server is being taken offline for maintenance, and those problems are being actively addressed.  We will share an ETA for repair once one is known.

UPDATE, 1:35 p.m.: The server is once again running and we are verifying that services have been restored.

Posted in unplanned outage | Comments Off

AT&T spam blocking problem

Several of our customers have reported problems sending mail from their nccn.net e-mail addresses to various addresses supported by AT&T.  We’ve identified an issue sending to e-mail ending in the following domains:

pacbell.net
prodigy.net

Other domains hosted by AT&T, such as att.net and sbcglobal.net, are unaffected.

Symptoms

When you send a message, you receive an immediate rejection e-mail back stating that it did not go through, and the diagnostic code provided in that message looks like the following:

<someaddress@pacbell.net>: host al-ip4-mx-vip2.prodigy.net[144.160.235.144] said: 553 5.3.0 alph133 DNSBL:RBL 521< 23.253.230.108
_is_blocked.__For_information_see_http://att.net/blocks (in reply to MAIL FROM command)

Resolution

Unfortunately, the server which handles pacbell.net e-mail has chosen on their end to block messages from our outbound mail server. We have repeatedly notified AT&T of this problem. They have refused to address it and claim that this blocking is not occurring despite the evidence we continue to send them.

At this point if you need to send messages to an email address ending in pacbell.net we have two workarounds to offer:

1) Ask that friend/client for an alternate e-mail address, and send your message to them at that alternate address. No other ISPs are blocking us, and you can send to all addresses except for the ones that AT&T is blocking.

2) Log in to your e-mail account via our website at http://webmail.nccn.net. You can click on “Compose” inside web mail to create a message, and copy and paste the e-mail you want to send into that web screen. Our webmail interface passes through a different server that AT&T is not blocking.

At this time, because the blocking is occurring on their end and the final resolution is in AT&T’s hands, we do not anticipate a fix on any sort of reasonable timeline.  We will continue to pursue further workarounds and alternative long-term solutions.

Posted in service update | Comments Off

Server downtime

Due to the announcement of the so-called GHOST vulnerability, some of Spiral’s web hosting and e-mail services will be temporarily inaccessible tonight as patches are applied to affected servers.  The nature of GHOST requires that servers be halted and rebooted in order to apply the patch.

 

Update, 12:30 a.m. Jan. 28: Due to an unexpected issue with the upgrade on our primary hosting server, a number of our hosted websites suffered approximately a 3-hour outage.  However, the server is now secured and we’ve audited the sites to check for lingering issues.  If there are any problems with your website, please call us at (530) 478-9822, Ext. 1, so we can address it as soon as possible.

We will briefly delay the patch on our secondary hosting server until tomorrow, so we can secure the server without such extensive downtime.

Update, Jan. 29: Our secondary hosting server was patched tonight as well, and no further service downtime is anticipated.

Posted in Intrusive maintenance | Comments Off

All systems normal

While recent storms have battered the area and increased AT&T’s response time on telephone line repair (affecting our DSL customers secondhand), all Spiral services are currently up and running at 100% capacity.

We have also completed our migration of services to the cloud, so email and web services are no longer vulnerable to local power outages, and offer significantly better bandwidth and response time (with no increase in cost to our customers).

As always, if you are experiencing problems with your Spiral services, you can give us a call for support at (530) 478-9822, Ext. 1.

Posted in service update | Comments Off

Brief inbound e-mail outage

Tonight’s transition of our inbound e-mail’s spam filtering server to a connection with faster throughput, which was meant to be transparent, ran into an unexpected snag as the server was being redeployed.  For a brief period around 7:00 p.m. Dec. 1, some inbound messages for Spiral’s e-mail customers were rejected by our mail server.

After fixing the error, we instructed the spam filter (which correctly passed the mail through) to redeliver all affected messages.  No mail should be missing.  However, a small number of senders received bounce reports for the messages they sent during that time window (the messages which we later ensured did pass through).

If you have an e-mail address hosted by Spiral and are missing any inbound e-mail from the evening of Dec. 1, or have been informed by a third party that a message did not get through to your nccn.net/spiralemail.com/sierraemail.com address, give our office a call at (530) 478-9822.  We can examine the spam filter’s records of your inbound e-mail and ensure that everything reached you which should have reached you.

Posted in unplanned outage | Comments Off

Email migration complete

The server on which NCCN.net e-mail is hosted has now completed tonight’s scheduled migration to the cloud.  The move was uneventful, all data carried over successfully, and access has been tested via both direct login and our NCCN.net webmail client.

If you have an e-mail address ending in @nccn.net, you should already be noticing a dramatic difference in server speed and accessibility (especially when accessing mailboxes with thousands of stored messages).  We’ll continue to upgrade the cloud server over the next few months, adding more storage space for your e-mail and more features to the webmail client.

If you have any problems with, or questions about, the transition, give our technical support team a call at (530) 478-9822, Ext. 1.

Edit: If you are having problems connecting to the server when sending, but are receiving mail properly, make certain that you are using port 587 for your outgoing mail (we have received many reports that port 25, which we don’t officially support, worked on the old server but not the new one).  You can call us for technical support if you would like help changing this setting.

Posted in planned maintenance, service update | Comments Off

“Shellshock”: Major Internet-wide security hole

A critical security vulnerability (now known as “Shellshock”) was announced today, affecting a system shell used by the vast majority of Internet servers.  The vulnerability allows remote hackers to run code on a system without authorization (somewhat like having a virus, but without any malicious code being installed).  We are taking this very seriously and are in the process of applying security patches to all Spiral services.

According to many reports, Mac OS X is also vulnerable — Macintosh users need to keep an eye open for security updates from Apple, and install them immediately once a patch is released.

Even Android smartphones may require a system update to address this, though it’s not known yet the extent of that vulnerability.

We will update this post as systems are secured and more information about Macintosh patches is released.


Update, 9/25: All Spiral servers were secured last night before the first exploits were published, though a smaller secondary vulnerability was identified and we are waiting for further official patches before declaring the issue fully resolved.

The primary concern for Macintosh users would be if you are running external services on your machine, such as a web server.  Although some websites are suggesting an immediate recompile of the Mac OS X bash binary, it is likely that the average Mac user should wait for an official patch from Apple rather than risk damaging their own system.

Best available information is that smartphones are not vulnerable out of the box, but specific applications may install or use the bash shell that’s at the root of the problem.

The vulnerability does not appear to exist in the Pace DSL modems that we sell, but we are currently working to confirm that.


Update, 9/30: Apple has released a patch for Mac OS X (versions 10.7 and up), which fixes most of the vulnerabilities identified.  If you use a Macintosh computer, make certain all recent software updates have been applied.

The final patches to secure against all known variations of the Shellshock bug have been applied to all Spiral servers, and we are continuing to monitor for related outbreaks or exploits.

Posted in Uncategorized | Comments Off

Outbound e-mail interruption

This afternoon, as we completed the process of decommissioning our local name server, a misconfiguration in our NCCN.net e-mail server caused outbound messages to briefly fail and inbound messages to be halted and queued for later delivery. The issue has been fixed; outbound messages are being correctly sent as of approx. 5:15 p.m., and the spam filtering system is currently working on processing the queue of inbound messages.

All other aspects of the DNS transition went smoothly. If you have DSL or Fusion service with a static IP address and have not yet updated your nameservers, click here for the support article.

Posted in unplanned outage | Comments Off