Data Lakes for Smarter Associations

Open Source is Winning the Internet. Join us to be the BEST regardless of culture or country.

What is a “data lake”?

Data lake – it’s a big, partially structured and differently structured set of data from your association that you can use to answer interesting questions. Think of it like throwing everything in your pantry into one magical cauldron and it comes back with answers. Of course be careful what you ask, as SECURITY IS ALWAYS JOB 1! But it’s cool.

This is how AWS visualizes it:

Source: https://aws.amazon.com/big-data/datalakes-and-analytics/what-is-a-data-lake/

Having been personally involved in politics, analytics, electrical engineering, programming, robotics, public relations and media strategy not to mention the whole entrepreneurship thing, I must to start with a warning; do NOT go overboard.

Luckily Tendenci as your AMS makes this a whole lot easier with all of the ways to import and export data (or your whole database.) And pretty much every report has a structured export as well, from custom forms, donations, event registrations, memberships, you name it. You have FULL ACCESS to YOUR DATA in both flat files and in the actual database schema. For example:

I like this quote from wipro:

If the data in a lake is not well curated, then it may turn into a data swamp, flooding an organization with information.

https://www.wipro.com/en-US/analytics/five-best-practices-to-keep-your-data-lake-healthy/

You can see more screen shots of the built in reporting within Tendenci. Yet let’s be realistic, you are integrating data from many sources and a Data Lake including but not limited to your data on Tendenci might be just the thing. Contact us for more, because we LOVE DATA!

And we’ve got over 20+ years of experience to back it up.

Why not make 2020 the banner year to grow your membership and donations? Because that’s what we THRIVE ON! YOUR SUCCESS!

Tendenci AMS API Integration

Python Rules

OCT 2023 UPDATE: For information on accessing Tendenci data via API please see our GitHub discussion at https://github.com/tendenci/tendenci/issues/1023

For more on The Open Source AMS integration via API visit our AMS API Helpfile

Tendencithe Open Source AMS is unique in that it is fully open source. However at times people would prefer to use an API to pull specific information. For that Django has several API integrations for your Association Management System such as:

django-tasty-pie is a REST based API to your AMS
https://github.com/django-tastypie/django-tastypie
The Django Rest Framework is also something the Tendenci community has been discussing switching to it as well.
http://www.django-rest-framework.org/

API’s aren’t mutually exclusive after all, right? You have options.

There are legitimate reasons to use an API. Examples include integration between a legacy mainframe system, ecommerce, or a development team that has chosen a different platform such as .NET or PHP.

Tendenci doesn’t meet all of the functional requirements for everyone by design. Instead we work with great technology like machine learningThe open AMS community isn’t focused on reinventing the wheel. It just doesn’t make economic sense for a non-profit, or even a for profit company, to reinvent Amazon.com or Ebay.com. This is particularly true if you are causes-based association or non-profit given the expense.

Does Tendenci AMS work with other providers? Absolutely. Any provider with an API or that supports SSO or RSS or has their own technology like google tag manager.

Non profits don’t have money to waste. Therefore we aligned our product to major industry supported technology.

Our technology stack as of 2018 is:

  1. Tendenci
  2. Django Web Framework
  3. Javascript and jquery
  4. Bootstrap CS
  5. Python Programming Language
  6. Postgres Database with GIS
  7. Docker Containers
  8. Ubuntu

For more on The Open Source AMS integration via API visit our AMS API Helpfile or read up on everything Tendenci Works With. Or if you aren’t into open source, there are definitely alternatives to Tendenci.

If you do pick an alternative, we suggest you consider Security FIRST and go from there.

Why Tendenci Chose Python over PHP

Note: this is a repost from the eschipul.com blog and also lives as a help file on Tendenci.

This blog is a WordPress blog written in PHP. And WordPress, which is written in PHP is a great platform when secured properly.

So why did our team choose to rewrite Tendenci Open Source and in the Python Programming language? It is a question I get asked a lot. We’ve never been a company that likes to talk in the negative if at all possible, yet it is important to talk about the megatrends going on given we work with associations and nonprofits.

Python Growing in Academia
Why Python instead of PHP for Tendenci

Source: https://www.upguard.com/blog/which-web-programming-language-is-the-most-secure

programming vulnerabilities
Vulnerabilities in each language

Source: http://info.whitehatsec.com/rs/whitehatsecurity/images/statsreport2014-20140410.pdf

security-report

Source: http://info.whitehatsec.com/rs/whitehatsecurity/images/statsreport2014-20140410.pdf

Popularity of a language is a trend, and what you want is as many developers familiar and liking the language of your open source project as possible. This means you have a better chance to have a secure web site and therefore a more secure future.

To be fair – as Disraeli said – “lies, damn lies and statistics” – so there is no one perfectly secure language any more than there is a perfectly “safe” hammer. There will always be operator error and programmers make mistakes.

So we’re not saying Python is perfect, and all of us have used most of the other languages on those charts at some point. We’re just saying we are pleased so many other programmers also like Python and Open Source. THAT is the best that can be done to secure your future online. Secure code that you can examine yourself and even host yourself!

Addendum: As I post this on the Tendenci Blog. Given we focus on non-profits, associations, memberships, education, medical, religious – basically the do-good cause-based organizations, I believe it is particularly important that the project is as transparent as possible. Sometimes it is healthy to inform everyone of WHY we made a decision seven years ago. Python was the right call.

Update on Tendenci 5 sites

To our clients on the Open Source Tendenci 5, and the brave clients volunteering to beta test with us on Open Source Tendenci 6 (which I haven’t even had a chance to blog about yet) – all of y’all are still online, have had zero downtime and remain rock solid. Linux and Django and Containers are definitely proving how much stronger they can make Tendenci. This is done by design and made possible by virtue of the flexibility and low cost associated open source in the cloud. It is achieved through isolation, portability and flexibility. I hope you are not frustrated by our team being laser focused on helping our long time clients who experienced outages. I apologize for the slower response time. I know you are missing reports and other items that were there in T4; they will return to being my focus once all of our data centers are fully back online regardless of technology.

Further I am aware of the fact this has thrown numerous projects wildly behind on their timelines and disrupted you as well. All things considered, if your site was offline, you would demand the same from us – to focus on bringing everyone back up.

Ethically, we (Tendenci) must stay the course and get these sites functional. Even now I feel guilty taking the time to write this instead of working on the technical details. I also know people need to know we have a plan (we do) and there is an end in site (there is) and that it will be a success (it will be). And that we have learned from it (we have).

To our Tendenci4 legacy clients on the Microsoft platform, you are and have been MY TOP PRIORITY and the top priority of the entire team. We knew the Internet had changed, just perhaps not how much it had changed in the category of zero day types of threats. See next post.

 

Tendenci Templating – we can Improve

Tendenci 4 used basic Dreamweaver DWT files so designers could see everything they were doing. Then we used very strongly typed and very exact html comments along with Python to chop up the template to integrate it with the code.

The benefits were huge in that designers could design, programmers could program. Life was good. Well, almost, there was the whole 2002 use of tables because Netscape and IE were fighting and munged up generated HTMl inside of the script blocks rendered out. Very non MVC but it worked. And it allowed for amazing designer freedom.

Django – Tendenci has been a team effort since 2001 and the big jump came in 2009 when we started the rewrite in fully open source code (old news, I won’t bore you with it, but you can google it.). Django, a web framework written in Python, replaces the custom written framework we built in Tendenci 4 from scratch (hint – avoid writing frameworks from scratch. It makes no sense anymore. Time changes things.)

The problem? Django templates are out of date, they use too much magic (how many times does a designer add a code block and forget the templatetag? Or more common, remove a code block and leave the templatetag out of forgetfulness or fear.

And the worst part – you CAN’T SEE IT unless it is rendered. No more Dreamweaver or any other wysiwyg tools. We take visual people and thrown them into a text based world. For comparison, this would be like coding in Python in Photoshop. It doesn’t make sense.

I wish this post had a happy ending. It is a happy goal. A worthy goal. But the state of affairs with Django and Templates remains stagnant at best as explained here:

https://speakerdeck.com/mitsuhiko/lets-talk-about-templates

We can improve and the future will be brighter when we can just drop in a bootstrap-theme.css file and change our sites as easily as css zengarden.

Tendenci Open Source Donor Management Roadmap

Tendenci has come a long way since it was started in 2001. I didn’t have a choice back then so thus began (at the time – we are now open source) a proprietary system. But we work mostly with Associations and Non Profits. They/You (and I can attest first hand after volunteering with several nonprofits over the years) don’t want proprietary – you want OPEN SOURCE!

So what is our roadmap for Tendenci, now completely open source, for donor management?

I have to say we weren’t feeling the urgency on the donor management part until recently. Why? Because Blackbaud (Nasdaq BLKB) acquired Convio and removed the only viable option. And now they are shutting down Common Ground. Not cool. (Side bar: Word is Convio used to advertise “Common Ground, because we’re not Raiser’s Edge.” If that is true or not, it’s still funny and was always a comment that came up at NTEN or Techsoup meetings over the years.)

Tendenci is written in Django and Python and will remain open source. We are adding a basic GPL CRM with modifications for donor management first. More importantly we are integrating Tendenci with the Salesforce Foundation and the Non-Profit Starter pack through their open source connectors and well documented API.

The team at Tendenci is working feverishly to get full Tendenci-Salesforce integration done because we need it too!

We are prepared and qualified. We have a former employee who now works at Salesforce, we use Salesforce ourselves, and quite frankly they seem to care about the non-profit world. We at Schipul are “all in” as they say as are Tendenci powers the majority of our clients are non-profits.

What can you do now to get ready?

If you want to get ready to fully integrate your Tendenci site, hosted with us or self hosted with the open source community version, a good jumping off point would be to read up on the great work the Salesforce Foundation does for Non Profits. Sign up with the Foundation to get your free licenses even if you just want to play around with it.

I’ll try to keep everyone more up to speed on our roadmap, it’s been a crazy year. But make no mistake, we are building an open system that will reduce the costs of advocacy and non-profit associations and foundations by a factor of 10 compared to Raiser’s Edge. There is hope. (Programmer? – join us on github! http://github.com/tendenci/tendenci ) And in case you can’t tell, I’m excited about this!

Post Django Dash 2012 Recap

We did it. Django Dash for our second year in a row. A little different, but still memorable.

From home

This year we spent the majority working from the comfort of our own homes.

Thanks to Schipul and our decision to move toward a remote work lifestyle we were able to easily face this year’s competiton in style aka in our jammies.

If anything this saved us valuable travel and setup time.

Lessons learned

I said this last year, but I’ll say it again this year; because apparently we did not learn our lesson.

Make as many decisions as possible before the competition. Think roadmap or dare I say clipboard of fun.

One of our greatest strengths is our team. We work together every workday, this competition was no different. I can only imagine the stop-and-go speed of competing on a team that doesn’t know each other.

Not the time to learn

This year I spent some time on two things I’ve only spent a couple of minutes on in the past. The Twitter Bootstrap project and Class Based Views. While my colleagues spent their time learning about Google authentication and the interim their experiencing as they adopt new technologies.

Competition time is definitely not the time to learn new things. It’s just so hard not to. You find yourself inspired and when inspiration strikes all you want to do is strive, learn new things and create.

In the case of Google authentication, it had to be learned.  Our project was dependent on it, as always; it’s amazing what you can do when you have to.

I don’t foresee this habit waning any time soon. If anything I look forward to it. I learned a lot of useful things this weekend and I’m left wanting more. Give me that feeling anyday.

Veering from the original mission

Early Sunday morning we found ourselves having to make a choice. A choice between accumilating more points by submitting more commits and focusing on specific code requirements such as standards and creating tests. Or making a product that might actually provide some value to many in the near future.

We chose the latter. The idea of our project actually being useful at more than just collecting points is an honor. With this in mind we refocused and put effort into submitting a finished product that’s worth demoing.

We’ll be demoing our finished 48 hour project to the office and get our first ouside perspective. No matter what people say I’m not-so-secretly wishing we can keep up this momentum and continue improving our project.

What did you build, tell me already!?

Without getting into too much detail – at this point in time – it’s best summarized here. http://theoldmail.com

You can sign up for the site now and take it for a spin. Keep in mind that this was 48 hours of code. You might find some quirks and so-called missing features.

What about the competition?

It’s been said that we get our results some time this week; but as I mentioned before we’re more excited about the project itself and what it can bring to others.

It’s open sourced

One of the rules of the Django Dash competition is that the project itself must remain open sourced. So feel free to take a glance at our code on github.com and fork the code if you’d like to start contributing.

Thanks to the programming giants who walked before us

Giants. Giants I tell you.

Yes, I’m excited about Tendenci going open source. But first – THANK YOU to the giants who walked before us to make this possible.

As a programmer, granted I haven’t been in the code much the last 5 years doing the whole “running the company thing”, but I haven’t forgotten how important it is to give credit where credit is due. We just pushed Tendenci 5.0 live on github yesterday. As far as I know, Tendenci is the only “open source CMS system built specifically for nonprofits” and I could add “written on the Django framework and Python.”

That is what makes open source so cool. As David Geilhufe told me today when we bumped into each other at NTEN, “welcome to the open source community. it took you a while but you got here.” David’s right.

Tendenci 5 was a complete rewrite and took over 3 years to complete.  I have said thanks to our programmers numerous times. But what giants’ shoulders did we stand on? Quite a few. Tendenci would not have been possible without the original committers on Django. So a RESPECTFUL tip of the hat to these trail blazers. #respect #thankyou

  1. Adrian Holovaty
  2. Simon Willison
  3. Jacob Kaplan-Moss
  4. Wilson Miner

And the list of brilliant committers goes on. Because it is a community. These people enabled us to give. I respect that.

To put it all together, Tendenci is a full web application. It is written on a framework called Django which is “the web framework for perfectionists with deadlines.” Django is written in a programming language called Python. Python was created by the amazing Guido van Rossum. I have never met Guido. But I know his brain is absolutely amazing and that our current business model would not be possible without Guido. Thank you Mr. Rossum. And thank you for everyone who contributes to the Python project.

We have a lot to learn still. And we are studying and learning as fast as we can. Any help is appreciated. But first and foremost, thank you to all of the programmers who walked before us and made our current path possible. ~Ed