ctop – measure container cpu utilization like htop

Developers and programmers are frequently (ok, almost always) asked to accomplish the impossible yesterday. So this post is for the Tendenci developers and anyone else who uses docker containers, cgroups, jailed name spaces or similar.

Situation: You have a server that is spiking when it previously did not.

Let’s just assume you already have something like OSSEC and the ElasticSearch Stack  (ELK Stack) installed and are using a WAF/IDS/IPS endpoint. You are on top of your game. You see the errors from writing to the file system in dockers using the overlayfs file system (please no aufs, just don’t.) How to diagnose it:

“htop” is very good at showing you the issue. It (htop) is also frequently replaced by malware so double check yourself with “ctop” which most variants of common malware omit. Regardless, in this case, we can clearly see we have a stuck process. Enter “ctop” (open source like Tendenci at https://ctop.sh/ and on github at https://github.com/bcicen/ctop .

Running ctop you can quickly identify the container that is using the resources and then enter that container for further trouble shooting. “ctop” look like this:

The solution to a container over utilizing its resources is up to you and your developers. ctop is however a great way to zero in on at least which container is the problem.

In our case, a quick stop/start of the container removed the load and allowed us to do more debugging to figure out the cause. Tendenci is a mature and large codebase for association management (AMS Software) so it’s an iterative process to zero in on issues. And it can be done with the right tools.

Happy Container Utilization

This is what one of the Tendenci Cloud docker servers looked like after  debugging and killing the process causing the problem. “Yes” of course there is no replacement for “grep”. But with containers the debugging is a new art even for experienced programmers.

Hopefully this is helpful for all of the open source self-hosted Tendenci – the Open Source AMS self install developers using an AMS with 75+ languages out there.

And if you are a Python/Django developer – fork Tendenci open ams on github!

#peace

The Incredible Growth of Python – StackOverflow

growth of python programming language

Python, the language used to program TendenciThe Open Source AMS, continues it’s meteoric rise in the world of developers. And where the developers go is where the rest of us go. Thus Python’s rise matters. And it benefits every Tendenci user, self hosted or hosted with our small company (same software either way).

IEEE Spectrum rates the languages by its readers as follows:

Python has continued its upward trajectory from last year and jumped two places to the No. 1 slot, though the top four—Python, CJava, and C++—all remain very close in popularity.

StackOverflow, a go-to site for pretty much every programmer and sysadmin out there, has a new blog post up on the incredible growth of the Python Programming Language. Python is of course the programming language used in Tendenci – The Open Source AMS. From the Stack Overflow post:

June 2017 was the first month that Python was the most visited tag on Stack Overflow within high-income nations. This included being the most visited tag within the US and the UK, and in the top 2 in almost all other high income nations (next to either Java or JavaScript). This is especially impressive because in 2012, it was less visited than any of the other 5 languages, and has grown by 2.5-fold in that time.

They have numerous charts to back up the data, but these two in particular paint a telling picture.

From Stack Overflow – the current tag questions viewed:

Growth of Python Programming Language
StackOverflow – The Incredible Growth of Python

Perhaps even more impressive is the projection on the continued growth of Python. Just WOW!

growth of python programming language
Python – Incredible growth with developers

The above graphs should give you confidence in your choice of using Tendenci as your AMS as the developers are not only there, but growing. Given Tendenci is fully open source (this is different from “free trial” AMS systems which are NOT actually FOSS (Free and Open Source Software). Wikipedia describes the difference as:

(FOSS means) anyone is freely licensed to use, copy, study, and change the software in any way, and the source code is openly shared so that people are encouraged to voluntarily improve the design of the software.[3] This is in contrast to proprietary software, where the software is under restrictive copyright and the source code is usually hidden from the users.

Many of our competitors who are NOT Open Source and not true FOSS which can sometimes confuse people. I’ll do a future post on examples of sometimes misleading representations by AMS systems that are not “actually” Free and Open Source (FOSS)  as defined on Wikipedia.

The good news is with the growth of Python, it only make sense that developers will look at and many will join in to help the community improve the software as they join associations themselves.

We’ve written about why we chose Python over PHP to develop Tendenci open source several times. Correctly choosing the open source stack gives us, and everyone in the community, confidence to see the trends predicted correctly. It wasn’t rocket science – we just listened to our team, we listened to younger developers, and most importantly we listened to our clients on what the future was/is going to be.

And associations are kind of a big deal and they can’t use minimum viable products.

Why are associations unwilling to accept apps that meet only minimal requirements? Um… because they started as Guilds and go back to Medieval times. From Britannica on Guilds and Trade Associations:

Guildalso spelled gild , an association of craftsmen or merchants formed for mutual aid and protection and for the furtherance of their professional interests. Guilds flourished in Europe between the 11th and 16th centuries and formed an important part of the economic and social fabric in that era.

and Britannica goes on….

… associations are known to have existed in ancient Rome, however, where they were called collegia. These craft guilds seem to have emerged in the later years of the Roman Republic. They were sanctioned by the central government and were subject to the authority of the magistrates.

This is a huge topic of course. Just know that Tendenci is the ONLY top ranked AMS system that is truly FOSS. Unlimited admins, users, contacts – you can self host or if hosted with us we only charge for processing power. Got 1M users and contacts and 50 admins? No problem. And the growth of Python assures your continued freedom from vendor lock-in no matter what.

#peace and happy (Python) programming y’all!

Communications Needs a Town Square

Communication needs a town square. Step 1 was to implement a forum independent from our site. Encrypted (of course) and endorsed, and with it’s own unique karma.

 

Talk to us. Tell us what we can do better. Group listen – a challenge to all of us is to listen as a group. To truly listen.

community-tendenci-comAnd the software needs direction. Our job is to listen to and foster the community of Tendenci users across the globe while respecting that ultimately we, they, you, them, us, are all independent individuals and organizations, cultures and people, nonprofits and companies, we speak different languages, the only thing in common with the global tendenci community is our diversity. Just as the only constant is change.

We wouldn’t be cause related software developers or working at cause related companies that aspire to the #openeverything OSCON type of ethos to begin with.

First – it is to serve.

To achieve our communication goals we are using some amazing open source software based on django and postgres just like tendenci of course – The Misago Open Source Discussion Forums is used to power the new https://community.tendenci.com site to provide a “place” for that dialog to take happen. Developers are welcome to continue posting issues on github of course, clients who prefer to have us (or you!) manage their tendenci hosting will submit billable tendenci support requests, but there is something different about a forum that is toned down and not quite as public. I can’t quite explain why.

Join us?