Year: 2014

In migrating almost all of the remaining sites from Windows 2003 to Windows 2012 to ensure everyone is protected we ran into some issues implementing SSL on some of the sites.

Sites on T4 that were being migrated may experience connection outage anticipated through tomorrow afternoon while we implement SSL on these sites.

T4 sites that were already migrated last week experienced an outage of up to 2 hours today while the new firewall was being implemented. It was rolled back because of too many false positive blocking access to the sites. Those site have been brought back up with the rollback of the firewall.

 

 

In migrating almost all of the remaining sites from Windows 2003 to Windows 2012 to ensure everyone is protected we ran into some issues implementing SSL on some of the sites. Because it is a shared server we are evaluating if we should apply encryption at the firewall level for everyone or continue to on the current path.

Regardless this has become an urgent matter as similar to to the initial incident a large number of sites are offline and it is imperative that we get them back on line as soon as possible.

We had a conference call scheduled for 3pm today to address questions for clients who are part of this migration. T

Given our CEO’s role as part of the security and remediation team, we need to cancel today’s talk and will hopefully reschedule for Thursday as events play out. Clients who are affected have received an email.

If you have not an email from our team and had been part of todays conference call please email support@tendenci.com so we can make sure that we provide you with updated information for the rescheduled call.

We do not expect this outage to be more than 24 hours given our experience restoring the initial group of sites on the Windows platform.

The data migration for server maintenance that was started last night is still in progress and some legacy sites may still be experiencing an outage while the migration finishes.

Please email support@tendenci.com if you have questions specific to your site.

 

With the recent denial of service attacks on some of our T4 sites, we are taking measures to increase security on all T4 sites.

Sites that have not yet been moved to the secured server environment will be moved starting tonight and continuing through Tuesday. December 30th. T4 sites that were affected by the DDoS attack on our servers in past weeks have already been moved to this new server.

We expect full functionality restored although clients on our T4 legacy software will experience some limitations on their sites over the next few weeks. These moves are to ensure security for all of our legacy clients not yet using the Linux based Tendenci 5+ software.

These security updates do not apply to any clients on Tendenci 5 in the Linux environment.

We have reached out to clients on T4 who will be affected by this move through contact email addresses on file with our team.  If you have not received a message from the Tendenci Team and have questions about your site please email support@tendenci.com

 

In Tendenci 4 and soon in Tendenci 5 you will have the option of a lower cost method of email tracking and having all emails come from your site or domain name. Two low cost options that we recommend are Mailgun, http://www.mailgun.com/ and Amazon Simple Email Services, http://aws.amazon.com/ses/.

Happy Holidays from the Tendenci Team!

Our offices will be closed from 1pm on Wednesday, December 24th through 8:30am on Monday, December 29th.

In the meantime,  leave us a message on our mainline at 281-497-6567 and we’ll get back to you as soon as possible next week or submit a ticket to helpdesk.tendenci.com.

 

 

We have been working to get all sites back up safely and securely that were affected by the recent DDoS attack on our T4 legacy platform.

As of this morning, we have 60% of sites that were affected back up online.

We have sent DNS information to a number of other clients whose sites will be restored once the DNS updates have been completed.  We are still working on restoring a few websites. Sites that were image heavy have taken longer to restore, as many images were corrupted during the attack on the system and removed during the subsequent security scans.

We are continuing to bring sites back up throughout the day

Currently sites are experiencing limitations related to sending emails, updating via the wysiwyg editor, and exporting/importing data. Today we are making steps to bring email back up online to the sites. When we have an update for timeline on email we will be posting to the blog.

Thank you for your patience as we work to secure your site and your data. If you have questions about the status of your site please submit a ticket to helpdesk.tendenci.com or email us at support@tendenci.com.

 

 

 

We have started the process of bringing sites that were affected by the recent DDoS attack back online with limited functionality.

So far we have successfully restored 10% of the sites that were taken out by this attack. We will continue to bring sites online today.

Currently, sites that are being brought up have limitations in sending email, making updates through the wysiwig editor, and exports/imports.  We are working on secure solutions for those functionalities and will have them working as soon as possible.

The Tendenci team has been individually contacting clients as their sites are ready for review.

We are still fighting through configuration errors on a few of the development sites and are contacting those clients to let them know we are working to fix these errors. Our team is working on a script to replace absolute links with relative links on sites which will correct linkage breaking issues. This should enable us to bring many more of the sites back up online.

If you have not yet heard from one of our team members and have questions on the status of your site, please submit a ticket at helpdesk.tendenci.com or email communications@tendenci.com.

A security update, and that’s what I get for being overly optimistic, but our initial scans found some issues. It is important to remember that a server that is on the Internet accepts inbound traffic on port 80 and 443, but it replies and can call out.

Our remediation plan called for building all new servers and porting the data, but if there is something that can call out once we open those ports then we are right back at ground zero. Possibly worse. And that is not acceptable.

I’ll let everyone know the minute we can let some testing begin. Maybe I’m being overly cautious at this point but given the situation, I believe it is warranted. Our current task is reconfiguring sites and we are having some challenges but are solving them one by one. (8.3 filenames get restored from backups for example and have to be removed again. That type of thing.) – Ed