AdoptOS

Assistance with Open Source adoption

Open Source News

Exiting the JVM ... !@#$%

Liferay - Wed, 12/12/2018 - 14:54

Today I experienced a great victory, but the day sure didn't start that way. In the days leading up to OSGI I spent a lot of time reading about it, listening to discussions, etc. I wanted to understand what the big deal was. Among the list of benefits, one stuck out most for me.

"OSGI does the dependency management, at runtime, for you. So if you have Module A that is dependent on Module B, and all of the sudden Module B vanishes, OSGI will make sure your environment remains stable by stopping any other modules (ie. Module A) that are dependent. This way, your user is not able to start interacting with A at the risk of 500 server app errors or worse."

WOW. I mean, I don't often experience this problem, but I like the idea of not having to worry about it! So when I started my journey with Liferay 7, I was keen to see this in action. I follow the rules. I'm one of those people. But from the beginning I would have some odd situations where if I removed the wrong module I would get the usual "Import-Package" error message etc, but I would also see this.

[fileinstall-/home/aj/projects/ml/liferay/osgi/modules][org_apache_felix_fileinstall:112] Exiting the JVM

I am on Linux. The process was still running, but the portal was not responsive. So the only way to fix this was to

  1. kill the process
  2. remove the module that caused the error (Module A)
  3. clear the osgi/state older
  4. start the server
  5. and redeploy in the right order

Ugh. And what a colossal waste of time. Over the past couple years I would ask people here and there about this error, maybe vent some frustrations, but everyone said the same thing -- "I've never seen that. I don't know what you are talking about". I figured -- pfft, mac users :)

But today, when it happened again I said to myself, there must be a reason -- so I started to search. As part of my search I took to slack. Piotr Paradiuk (who I had recently met in Amsterdam at DEVCON) was kind enough to rubber duck with me. If you don't know what Rubber Ducking is -- https://en.wikipedia.org/wiki/Rubber_duck_debugging

Piotr (and David actually, in another chat medium) both had the same reply "never seen it before". But over the course of try this and do that, Piotr sent me a link to github (https://github.com/liferay/liferay-portal/blob/65059440dfaf2b8b365a20f99e83e4cdb15478aa/modules/core/portal-equinox-log-bridge/src/main/java/com/liferay/portal/equinox/log/bridge/internal/PortalSynchronousLogListener.java#L106) -- mocking my error message :). But. after staring at this file, muttering under my breath, something caught my eye.

if ((throwable != null) && (throwable instanceof BundleException) && (_JENKINS_HOME != null)) { String throwableMessage = throwable.getMessage(); if (throwableMessage.startsWith("Could not resolve module")) { log.error(_FORMAT, "Exiting the JVM", context, throwable); System.exit(1); } }

More specifically --

private static final String _JENKINS_HOME = System.getenv("JENKINS_HOME");

:| ... I have jenkins running on my machine. Could that be it? So I stopped my instance and ran --

unset JENKINS_HOME

Restarted my portal and ran through the steps to try to reproduce. This time I get the Import-Package error stuff, but the JVM doesn't exit.

:|

So, the moral of the story. If you are like me and you want to have jenkins on your dev machine to play with and such -- virtualize it or run it in a docker container. Oh .. and add --

unset JENKINS_HOME

... to your LIFERAY_HOME/TOMCAT_HOME/bin/setenv.sh

Andrew Jardine 2018-12-12T19:54:00Z
Categories: CMS, ECM

Plan for Drupal 9

Drupal - Wed, 12/12/2018 - 13:38

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

At Drupal Europe, I announced that Drupal 9 will be released in 2020. Although I explained why we plan to release in 2020, I wasn't very specific about when we plan to release Drupal 9 in 2020. Given that 2020 is less than thirteen months away (gasp!), it's time to be more specific.

Shifting Drupal's six month release cycle

We shifted Drupal 8's minor release windows so we can adopt Symfony's releases faster.

Before I talk about the Drupal 9 release date, I want to explain another change we made, which has a minor impact on the Drupal 9 release date.

As announced over two years ago, Drupal 8 adopted a 6-month release cycle (two releases a year). Symfony, a PHP framework which Drupal depends on, uses a similar release schedule. Unfortunately the timing of Drupal's releases has historically occurred 1-2 months before Symfony's releases, which forces us to wait six months to adopt the latest Symfony release. To be able to adopt the latest Symfony releases faster, we are moving Drupal's minor releases to June and December. This will allow us to adopt the latest Symfony releases within one month. For example, Drupal 8.8.0 is now scheduled for December 2019.

We hope to release Drupal 9 on June 3, 2020

Drupal 8's biggest dependency is Symfony 3, which has an end-of-life date in November 2021. This means that after November 2021, security bugs in Symfony 3 will not get fixed. Therefore, we have to end-of-life Drupal 8 no later than November 2021. Or put differently, by November 2021, everyone should be on Drupal 9.

Working backwards from November 2021, we'd like to give site owners at least one year to upgrade from Drupal 8 to Drupal 9. While we could release Drupal 9 in December 2020, we decided it was better to try to release Drupal 9 on June 3, 2020. This gives site owners 18 months to upgrade. Plus, it also gives the Drupal core contributors an extra buffer in case we can't finish Drupal 9 in time for a summer release.

Planned Drupal 8 and 9 minor release dates.

We are building Drupal 9 in Drupal 8

Instead of working on Drupal 9 in a separate codebase, we are building Drupal 9 in Drupal 8. This means that we are adding new functionality as backwards-compatible code and experimental features. Once the code becomes stable, we deprecate any old functionality.

Let's look at an example. As mentioned, Drupal 8 currently depends on Symfony 3. Our plan is to release Drupal 9 with Symfony 4 or 5. Symfony 5's release is less than one year away, while Symfony 4 was released a year ago. Ideally Drupal 9 would ship with Symfony 5, both for the latest Symfony improvements and for longer support. However, Symfony 5 hasn't been released yet, so we don't know the scope of its changes, and we will have limited time to try to adopt it before Symfony 3's end-of-life.

We are currently working on making it possible to run Drupal 8 with Symfony 4 (without requiring it). Supporting Symfony 4 is a valuable stepping stone to Symfony 5 as it brings new capabilities for sites that choose to use it, and it eases the amount of Symfony 5 upgrade work to do for Drupal core developers. In the end, our goal is for Drupal 8 to work with Symfony 3, 4 or 5 so we can identify and fix any issues before we start requiring Symfony 4 or 5 in Drupal 9.

Another example is our support for reusable media. Drupal 8.0.0 launched without a media library. We are currently working on adding a media library to Drupal 8 so content authors can select pre-existing media from a library and easily embed them in their posts. Once the media library becomes stable, we can deprecate the use of the old file upload functionality and make the new media library the default experience.

The upgrade to Drupal 9 will be easy

Because we are building Drupal 9 in Drupal 8, the technology in Drupal 9 will have been battle-tested in Drupal 8.

For Drupal core contributors, this means that we have a limited set of tasks to do in Drupal 9 itself before we can release it. Releasing Drupal 9 will only depend on removing deprecated functionality and upgrading Drupal's dependencies, such as Symfony. This will make the release timing more predictable and the release quality more robust.

For contributed module authors, it means they already have the new technology at their service, so they can work on Drupal 9 compatibility earlier (e.g. they can start updating their media modules to use the new media library before Drupal 9 is released). Finally, their Drupal 8 know-how will remain highly relevant in Drupal 9, as there will not be a dramatic change in how Drupal is built.

But most importantly, for Drupal site owners, this means that it should be much easier to upgrade to Drupal 9 than it was to upgrade to Drupal 8. Drupal 9 will simply be the last version of Drupal 8, with its deprecations removed. This means we will not introduce new, backwards-compatibility breaking APIs or features in Drupal 9 except for our dependency updates. As long as modules and themes stay up-to-date with the latest Drupal 8 APIs, the upgrade to Drupal 9 should be easy. Therefore, we believe that a 12- to 18-month upgrade period should suffice.

So what is the big deal about Drupal 9, then?

The big deal about Drupal 9 is … that it should not be a big deal. The best way to be ready for Drupal 9 is to keep up with Drupal 8 updates. Make sure you are not using deprecated modules and APIs, and where possible, use the latest versions of dependencies. If you do that, your upgrade experience will be smooth, and that is a big deal for us.

Special thanks to Gábor Hojtsy (Acquia), Angie Byron (Acquia), xjm(Acquia), and catch for their input in this blog post.

Categories: CMS

Liferay Portal CE Clustering Returns

Liferay - Tue, 12/11/2018 - 16:35
Background

Last fall we reintroduced the option to use clustering in Liferay 7.0 by compiling a series of modules manually and including them in your project. We received a lot of feedback that this was a very cumbersome process and didn’t really provide the benefits we intended in bringing back clustering.  

Enable Clustering

Beginning with Liferay Portal 7.1 CE GA3, clustering can now be enabled just like previously in Liferay Portal 6.2 CE.  The official documentation covers the steps needed to enable clustering in GA3.  To enable clustering, set the property:

cluster.link.enabled=true

Successfully enabling clustering will result in the following message in the logs on startup:

------------------------------------------------------------------- GMS: address=oz-52865, cluster=liferay-channel-control, physical address=192.168.1.10:50643 ------------------------------------------------------------------- Other Steps We Are Taking for the Community

Restoring clustering is one of many steps we are taking to restore faith between Liferay and our community. Other initiatives to improve the developer experience with Liferay include: 

  • More GA releases of Liferay Portal CE to improve stability and quality
     
  • A single-core model for Liferay Commerce, making open source Liferay Commerce installations compatible with our subscription services
     
  • Better support for headless use of Liferay with front-end frameworks
     
  • Improved experience with documentation, including unifying all available documentation and best practices in a single place
     
  • Additional ways to use portions of Liferay Portal’s functionality (such as permissioning or forms) in non-Liferay applications (e.g. those built with SpringBoot)

We’ll be providing more information about each of these initiatives in the weeks to come. 

Conclusion

I want to thank you again for sticking with Liferay all these years as we’ve sorted out our business model (and sometimes made poor decisions along the way). I’m excited for the chance to invest in and grow our community in the coming months.  It is what sets us apart from the competition and is truly one of the most rewarding parts of what we do at Liferay.  We hope fully reintroducing clustering to be the first step towards achieving this goal and as always would love to hear what you think in the comments below.

Bryan H Cheung 2018-12-11T21:35:00Z
Categories: CMS, ECM

Talend Cloud: A hybrid-friendly, secure Cloud Integration Platform

Talend - Tue, 12/11/2018 - 16:21

As enterprises move towards massively scaled interconnected software systems, they are embracing the cloud like never before. Very few would dispute the notion that the cloud has become one of the biggest drivers of change in the enterprise IT landscape and that the cloud has provided IT a powerful way to deploy services in a timely and cost-effective manner.

However, the tremendous benefits that you’ve tapped into the cloud has to be balanced against the need to adopt, configure and manage services in a secure manner. Digital transformation with a non-secure cloud solution is impossible.

When we look at the integration world, it is often a fragmented place where applications and data may each exist on multiple platforms; IT teams are frequently supporting a plethora of on-premises and cloud apps while moving petabytes of sensitive data among those apps. Data leakage, corruption, or loss can be devastating to a business.

A cloud integration solution, such as Integration Platform as a Service (iPaaS), with subpar security mechanisms may make your sensitive data such as customer records, passwords, or personal information susceptible to data breach and misuse. One way to avoid this, is to choose an iPaaS that has robust security mechanisms such as data encryption, user and access protection, security certifications, and information security standards in place.

Talend Cloud is a unified cloud integration platform (iPaaS) that integrates data, people, and applications across all types of data sources: public cloud, on-premises, and SaaS applications to deliver fast and reliable big data analytics. Besides having natively inherited AWS’s security practices, the platform is built from the ground up with all pieces of the infrastructure puzzle taken into account to best secure each layer: physical, network, operating system, data base, and application. The entire stack is secure for each customer, partner, and workload. As part of the holistic security design, Talend Cloud has earned industry-leading security certifications to ensure that its platform meet exacting stands for a range of customers and industries.

Talend Cloud provides cloud platform security certifications such as:

  • SSAE 16
  • SOC 2 Type II certification
  • ISAE 3402 certification
  • Cloud Security Alliance Certification (Level 1)

As well as Compliance Certifications such as GDPR via AWS. Additionally, Talend Cloud provides security features and services such as data encryption, key management for tenant isolation and remote engine, bi-annual penetration test, SSO support for Okta, IAM management and many more.

To learn more about how Talend provides security in iPaaS, take a look at the complete list of powerful security features detailed in this Talend Cloud Security White Paper or drop us a line at  https://www.talend.com/contact/ for any questions.

 

 

The post Talend Cloud: A hybrid-friendly, secure Cloud Integration Platform appeared first on Talend Real-Time Open Source Data Integration Software.

Categories: ETL

Three issues that every organization must consider for optimal data usage

SnapLogic - Tue, 12/11/2018 - 16:03

Originally published on AG Connect.  Companies have an obsession with data. There is more data than ever before and companies generate large volumes of data about customers, finances, machines and social media to gain insights and determine business strategies. But it is difficult to extract value from those data. Many organizations have been so busy gathering data[...] Read the full article here.

The post Three issues that every organization must consider for optimal data usage appeared first on SnapLogic.

Categories: ETL

(Another) Liferay DEVCON 2018 Experience

Liferay - Tue, 12/11/2018 - 13:36

What a trip. From the dynamic format of the Unconference, to the Air Guitar competition, the opportunity to speak, and a lens into what's coming in 7.2, it was by far the best event that I have ever been to. I've been to many conference, but my first developer conference. It was so good that I forced myself to find the time to document the exprerience to share it with all of you. 

Day 1 - The Arrival

I arrived the day before to try to cope with the time change. If you arrive early morning and are trying to overcome jet lag with activities, don't start with the river cruise. David N. warned me, but I didn't listen. The warm cabin, the gentle rocking of the boat, the quiet -- all that was missing was a lullaby. David was right, I fell asleep for most of that ride. 

Day 2 - Unconference

I now understand why the Unconference has both limited seating and why it sells out so quickly. At first it feels a little bit like group therapy; everyone sitting in a circle, cues around the room for you to share and not be afraid to share. That's how I felt at least. But the dynamic agenda was inspiring. The participants decide the days topics. Anyone, ANYONE, can stand up, go to the mic, and propose a topic. The topics are then sorted and organized until you end up with an agenda. The discussions are open forums. You can share something cool you have done, ask a question, gripe about something that is bugging you, whatever you want. It's the perfect format for developers; after all we love to argue and we love to boast. Best of all, the leads from the various Liferay teams are present so it's a great opportunity to have your voice heard. For example, I proposed and led a discussion with a topic "Documentation: What is missing?". Cody Hoag, who is the lead for the docs team, was there and shared with us some of the challenges his team faces. He also took notes of the complaints, suggestions or ideas the participants had to digest and share with his team. For me the best part of the Unconference was the opportunity to connect not just with other community members but also the Liferay people from various teams. With only a little over 150 people in attendance it was a nice intimate atmosphere -- perfect for learning and sharing. As if the day and the sessions weren't good enough, the community event that was hosted by Liferay that evening was a blast. If you have not been to an unconference before, I would highly recommend you attend the next one! I certainly plan to. 

Day 3 - DEVCON: Day 1

If first impressions really are the most important thing, then the events team from Liferay nailed it. The venue for this year’s conference was a theatre. The main room (the Red Hall) was amazing and it was one of the first times I found myself at a conference sitting in a seat that I was sure didn't double as a torture rack. I know how silly that sounds, but when you spend most of your day sitting and listening to others talk, a comfortable chair is pretty important! 

The conference kicked off and Liferay didn't waste any time talking about 7.1. I could barely hold back a grin as Brian Chan mentioned BBSes and IRC (items I had included in my own talk that was yet to come). The keynotes and the sessions for the day were awesome -- though I made the same mistake I always make of attending a workshop, with no laptop. The worst part is the walk of shame 20 minutes in as you try to make a quiet (read: impossible) exit from the front of the room to the back. All the sessions I attended were very well done and I can't wait to see the recordings so that I can watch several sessions I was not able to attend. Session aside, there was a very rewarding personal moment for me -- receiving a 2018 Top Contributor award. It wasn't my first award, but who doesn't like to be recognized for the work they do? Actually, the best part of the awards piece was probably the guy to my right who was "picking it up for a friend" ... uh-huh :). 

The after party was chalk full of fun, fun that included and Air Guitar competition. What did I learn? Liferay doesn't just hire some exceptional developers, they also have some impressive air guitar skills! The combination of some awesome old school rock mixed with some all-in passionate head banging air guitar displays makes me think that there are several engineers and community members that might have had aspirations of a different career path in their youth. As if a great party and atmosphere wasn't enough, I also managed to meet Brian Chan (Liferay's Chief Technical Architect and Founder) -- the guy that started it all. He actually thanked me for the work I have done with Liferay. That was a little shocking actually. I give back to Liferay and the community because I feel Liferay and the community has done so much for me! As with the community event I also met some great people from all over Europe and beyond. I even met a Siberian who's name escapes me at the moment, but the shot of vodka he made me do in order to gain passage through the crowd to go home does not. 

Day 4 - DEVCON: Day 2

Day 2 for me was D-Day. My session was right before lunch. Having rehearsed more times than I could remember, most of my nervousness was gone.  My topic was: Journey of a Liferay Developer - The Search for Answers. The content was a community focused session where I decided to share with the group how I got involved with Liferay, the challenges I have faced along the way and how I overcame those challenges. I was a little uneasy since I wasn't really sure anyone would really care to hear my story, but when it was all over, people clapped, so it couldn't have been THAT bad :). All of this was validation that giving back, the hard work to prep, the courage to stand up and share, was worth it. It's something that I would gladly do again and something I would suggest everyone try at least once in their life. 

After my session I shook hands with Bryan Cheung (CEO of Liferay) and had a conversation with him for a few minutes around some of the pieces I highlighted in my talk. This is one of the many things I love about Liferay -- the fact that you can connect with people in their organization at all levels. You think Mark Hurd or Larry Ellison would ever take the time to sit down with me? Maybe, but I doubt it. 

Just like Day 1, the sessions were once again amazing. I learned about tools that were coming to improve developers experiences, I learned about integrations between Liferay and other platforms like Hystrix, and I watched some of my friends presentations discussing their challenges, including how to navigate the rough waters of an upgrade. The day ended with an exciting roadmap session where Jorge highlighted some of the things that are coming in 7.2 which, for me, was really exciting to see some of the potential features coming down the line.

Apart from all this though, I was amazed at how, in just a couple of days, I had covered material that would have otherwise likely taken me weeks or possibly months to learn. With all this information I was anxious to get home to both improve my skills, but also to share my experiences with others (developers and clients) so that we can all get the most out of our Liferay investments.  

Day 5 - Home Again

On your way out, before you check that bag? make sure you have your headphones -- I didn't. The silver lining though was that it gave me time to reflect.  9 hours to sit and reflect to be more precise (that's A LOT of reflection).  I found myself dreaming of the next conference that I might attend and topics I might propose with the hope that Liferay would once again give me the opportunity to share. My goal now? Continue, catching some of the content I missed during the conference (https://www.liferay.com/web/events-devcon), connecting with as many people in the community as I can and to make my way to Boston in 2019.

Hopefully this year goes well enough that I'll find myself back in Europe again next fall -- the question is, where? And will I see you there?

Andrew Jardine 2018-12-11T18:36:00Z
Categories: CMS, ECM

Data Matching with Different Regional Data Sets

Talend - Tue, 12/11/2018 - 11:22

When it comes to Data Matching, there is no ‘one size fits all menu’. Different matching routines, different algorithms and different tuning parameters will all apply to different datasets. You generally can’t take one matching setup used to match data from one distinct data set and apply it to another. This proves especially true when matching datasets from different regions or countries. Let me explain.

Data Matching for Attributes that are Unlikely to Change

Data Matching is all about identifying unique attributes that a person, or object, has; and then using those attributes to match individual members within that set. These attributes should be things that are ‘unlikely to change’ over time. For a person, these would be things like “Name” and “Date of Birth”. Attributes like “Address” are much more likely to change and therefore of less importance, although this does not mean you should not use them. It’s just that they are less unique and therefore of less value, or lend less weight, to the matching process. In the case of objects, they would be attributes that uniquely identify that object, so in the case of say, a cup (if you manufactured cups) those attributes would be things like “Size”, “Volume”, “Shape”, “Color”, etc. The attributes themselves are not too important, it’s the fact that they should be ‘things’ that are unlikely to change over time.

So, back to data relating to people. This is generally the main use case for data matching. So here comes the challenge. Can’t we use one set of data matching routines for a ‘person database’ and just use the same routines etc. for another dataset? Well, the answer is no, unfortunately. There are always going to be differences in the data that will manifest itself during the matching, and none more so than using datasets from different geographical regions such as different countries. Data matching routines are always tuned for a specific dataset, and whilst there are always going to be differences from dataset to dataset. The difference becomes much more distinct when you chose data from different geographical regions. Let us explore this some more.

Data Matching for Regional Data Sets

First, I must mention a caveat. I am going to assume that matching is done in western character sets, using Romanized names, not in languages or character sets such as Japanese or Chinese. This does not mean the data must contain only English or western names, far from it, it just means the matching routines are those which we can use for names that we can write using western, or Romanized characters. I will not consider matching using non-western characters here. 

Now, let us consider the matching of names. To do this for the name itself, we use matching routines that do things like phoneticize the names and then look for differences between the result. But first, the methodology involves blocking on names, sorting out the data in different piles that have similar attributes. It’s the age-old ‘matching the socks’ problem. You wouldn’t match socks in a great pile of fresh laundry by picking one sock at a time from the whole pile and then trying to find its duplicate. That would be very inefficient and take ages to complete. You instinctively know what to do, you sort them out first into similar piles, or ‘blocks’, of similar socks. Say, a pile of black socks, a pile of white socks, a pile of colored socks etc. and then you sort through those smaller piles looking for matches. It’s the same principle here. We sort the data into blocks of similar attributes, then match within those blocks. Ideally, these blocks should be of a manageable and similar size. Now, here comes the main point.

Different geographic regions will produce different distributions of block sizes and types that result in differences to the matching that will need to be done in those blocks, and this can manifest itself in terms of performance, efficiency, accuracy and overall quality of the matching. Regional variations in the distribution of names within different geographical regions, and therefore groups of data, can vary widely and therefore cause big differences in the results obtained.

Let’s look specifically at surnames for a moment. In the UK, according to the National Office of Statistics, there are around 270,000 surnames that cover around 95% of the population. Now obviously, some surnames are much more common than others. Surnames such as Jones, Brown, Patel example are amongst the most common, but the important thing is there is a distribution of these names that follow a specific graphical shape if we chose to plot them. There will be a big cluster of common names at one end, followed by a specific tailing-off of names to the other, whilst the shape of the curve would be specific to the UK and to the UK alone. Different countries or regions would have different shapes to their distributions. This is an important point. Some regions would have a much narrower distribution, names could be much more similar or common, whilst some regions would be broader, names would be much less common. The overall distribution of distinct names could be much more or much less and this would, therefore, affect the results of any matching we did within datasets emanating from within those regions. A smaller distribution of names would result in bigger block sizes and therefore more data to match on within those blocks. This could take longer, be less efficient and could even affect the accuracy of those matches. A larger distribution of names would result in many more blocks of a smaller size, each of which would need to be processed.

Data Matching Variances Across the Globe

Let’s take a look at how this varies across the globe. A good example of regional differences comes from Taiwan. Roughly forty percent of the population share just six different surnames (when using the Romanised form). Matching within datasets using names from Taiwanese data will, therefore, result in some very large blocks. Thailand, on the other hand, presents a completely different challenge. In Thailand, there are no common surnames. There is actually a law called the ‘Surname Act’ that states surnames cannot be duplicated and families should have unique surnames. In Thailand, it is incredibly rare for any two people to share the same name. In our blocking exercise, this would result in a huge number of very small blocks.

The two examples above may be extreme, but they perfectly illustrate the challenge. Datasets containing names vary from region to region and therefore the blocking and matching strategy can vary widely from place to place. You cannot simply use the same routines and algorithms for different datasets, each dataset is unique and must be treated so. Different matching strategies must be adopted for each set, each matching exercise must be ‘tuned’ for that specific dataset in order to find the most effective strategy and the results will vary. It doesn’t matter what toolset you choose to use; the same principle applies to all as it’s an issue that is in the data and cannot be changed or ignored. 

To summarize, the general point is that regional, geographic, cultural and language variations can make big differences to how you go about matching personal data within different datasets. Each dataset must be treated differently. You must have a good understanding of the data contained within those datasets and you must tune and optimize your matching routines and strategy for each dataset. Care must be taken to understand the data and select the best strategy for each separate dataset. Blocking and matching strategies will vary, you cannot just simply reuse the exact same approach and routines from dataset to dataset, this can vary widely from region to region. Until next time!

The post Data Matching with Different Regional Data Sets appeared first on Talend Real-Time Open Source Data Integration Software.

Categories: ETL

Paths to sustainability

CiviCRM - Tue, 12/11/2018 - 06:01

Last month I attended Sustain Summit 2018: an inspiring 'one-day event for open source sustainers' organised by Open Collective. It was great to spend a day with people who are working hard to tackle the sustainability problem. Some focused on their own projects, others looking more broadly across the open source ecosystem. The sustainability conversation has moved on significantly since the last time I looked, in about 2013, and there are lots of new ideas and projects with the potential to impact how we work.

Categories: CRM

Accelerate your machine learning (ML) projects with three new ML Snap Packs

SnapLogic - Mon, 12/10/2018 - 15:59

More and more organizations are searching for ways to drive business value with machine learning (ML), a practical form of artificial intelligence (AI). But conventional approaches to developing and deploying ML models are hampering their efforts. Traditional methods are slow, code-intensive, and require specialized skill sets. What’s more, data scientists and data engineers are forced[...] Read the full article here.

The post Accelerate your machine learning (ML) projects with three new ML Snap Packs appeared first on SnapLogic.

Categories: ETL

Events online training: December 11th

CiviCRM - Fri, 12/07/2018 - 17:16

Take your event management to the next level with this Cividesk on-line training session designed for current users of CiviCRM on Tuesday, December 11th  at 10 am MT/ 11 am CT/ 12 pm ET. This course is an excellent follow-up to the Fundamentals of Event Management class taught by Cividesk.

Categories: CRM

Hit the “Easy” Button with Talend & Databricks to Process Data at Scale in the Cloud

Talend - Fri, 12/07/2018 - 15:00

The challenge today for big data is that 85% of on-premises based Big Data projects fail to meet expectations and over 2/3 of Big Data potential is not being realized by organizations. Why is that you ask? Well, simply put on-premises “Big Data” programs are not that easy. 

Your data engineers will need to be proficient in new programming languages and architectural models while your system admins will need to learn how to set up and manage a data lake. So you’re not really focusing on what you do best and instead paying top dollar for data engineers with the programming skills while spending (wasting) a lot of time configuring infrastructure – and not reaping the benefits of a big data program.

In short, making big data available at scale is hard and can be very expensive and the complexity is really killing big data projects.  

Welcoming modern data engineering in the cloud

Data engineers ensure the data the organization is using clean, reliable, and prepped for whichever use cases that may present themselves.  In spite of the challenges with on-premises “big data,” technologies like Apache Spark have become a best practice due to its ability to scale as jobs get larger and SLA’s become more stringent.  

But using Spark on-premises as we’ve highlighted is not that easy.  The market and technologies have come to an inflection point where it is agreed that what is needed is the ability to:

  1. Eliminate the complexity of system management to lower operations costs and increase agility
  2. Have automatic scale up/down of processing power, to grow and shrink as needed while only paying for what you use
  3. Enable a broader set of users to utilize these services without requiring a major upgrade in their education or hiring expensive external expertise

To simplify success with big data programs, market leaders have moved from an on-premises model to a cloud model.  Cloud based environments offer the ability to store massive volumes of data as well as all varieties (structured to unstructured). Now what is needed is the ability to process that data for consumption in BI tools, data science, or machine learning.

Databricks, founded by the original creators of Apache Spark, provides the Databricks Unified Analytics Platform. Databricks accelerates innovation by bringing data and ML together.  This service solves many of the hard challenges discussed above by automatically handling software provisioning, upgrades, and management.  Databricks also manages the scaling up and down to ensure that you have the right amount of processing power and saving money but shutting down clusters when they are not needed.  By taking this workload off the table for their customers, this allows those customers to focus on the next level of analytics – machine learning and data science.

While Databricks solves two out of three of the big challenges posed, there is still the third challenge of making the technology more accessible to “regular” data engineers that do not have the programming expertise to support these massively parallel, cloud environments.  But that is where Talend comes in.  Talend provides a familiar data flow diagram design surface and will convert that diagram into an expertly programmed data processing job native to Databricks on Azure or AWS.

The combination of Databricks and Talend then provides a massively scalable environment that has a very low configuration overhead while having a highly productive and easy to learn/use development tool to design and deploy your data engineering jobs.  In essence, do more with less work, expense, and time.

For further explanation and a few examples, keep reading….

Example use case

Watch these videos and see for yourself how easy it is to run a Spark Serverless in the Cloud.

Movie recommendation use case with machine learning and Spark Serverless

 

  Create and connect to a Databricks Spark Cluster with Talend

 

Click here to learn more about serverless and how to modernize your architecture?

Check out our GigaOM webinar with Databricks and Talend to learn how to accelerate your analytics and machine learning.

The post Hit the “Easy” Button with Talend & Databricks to Process Data at Scale in the Cloud appeared first on Talend Real-Time Open Source Data Integration Software.

Categories: ETL

Final curtain call for PrestaBox

PrestaShop - Fri, 12/07/2018 - 10:00
Following the end of the offer’s marketing in 2015, PrestaShop hereby announces the withdrawal of PrestaBox, its first Cloud offer, on March 07, 2019.
Categories: E-commerce

5 customer retention tips to grow your business

VTiger - Fri, 12/07/2018 - 07:36
Whether you are a start-up or a Fortune 500 company, retaining customers is your magic potion to run a profitable business. While customer acquisition is all about attracting new customers to grow business, customer retention focuses on keeping customers happy and doing more business with you. Why should you focus on customer retention? While customer […]
Categories: CRM

Gartner Application Strategies & Solutions Summit recap: How Wendy’s stays competitive

SnapLogic - Thu, 12/06/2018 - 17:46

The SnapLogic team joined thousands of business and technology leaders recently at the Gartner Application Strategies & Solutions Summit (APPS) in Las Vegas. With dozens of workshops, breakout sessions, and meetups, attendees and vendors alike had several opportunities to learn from and network with Gartner analysts, industry experts and peers on API management, AI, DevOps,[...] Read the full article here.

The post Gartner Application Strategies & Solutions Summit recap: How Wendy’s stays competitive appeared first on SnapLogic.

Categories: ETL

It's a nice day for a white menu...

CiviCRM - Thu, 12/06/2018 - 14:20

It's a nice day to... LEEEEExIIIIIIIM!

Categories: CRM

CiviCRM 5.8.0 release

CiviCRM - Thu, 12/06/2018 - 08:57
CiviCRM version 5.8.0 is now ready to download.  RELEASE NOTES: Big thanks to Andrew Hunt from AGH Strategies for putting up together release notes for this version.  The release notes for 5.8.0 can be accessed here.   SPECIAL THANKS:
Categories: CRM

KNIME Analytics Platform 3.7 is now available!

Knime - Thu, 12/06/2018 - 05:30
KNIME Analytics Platform 3.7 is now available! admin Thu, 12/06/2018 - 11:30

We’ve just released the latest versions of KNIME Analytics Platform and KNIME Server and here’s a quick summary of what’s new.

Some of the highlights in this release are a number of new interactive views (check out the new Tile View* and Heatmap), new integrations allowing KNIME workflows direct access to Google Drive and Tableau’s Hyper format, and a number of new statistical tests. New XGBoost nodes add access to this well known high performance machine library, while a more hidden gem is the completely new layout editor for composite views of metanodes, which allows you to build your own library of personalized view nodes.

There are also new integrations with Jupyter Notebooks and PySpark. Now you can import and run code from Jupyter notebooks directly in your KNIME workflow. Adding Python to your big data workflows is also now possible, allowing you to seamlessly mix and match Python with existing nodes from the KNIME Extension for Apache Spark.

KNIME Server offers new automation and management services for data science and IT teams productionizing workflows and data science applications. Now you can automatically scale your infrastructure with our Distributed Executors and automate remote workflow editing and control. New management services are also available for centrally adding customizations to KNIME Analytics Platform users.

All the release details are summarized here, as well as in the detailed changelog. However, the best way to check it out is by trying it for yourself. Open KNIME and go to File -> Update KNIME to get the latest version, or download and install it here.

Remember to mark your calendars for the KNIME Spring Summit in Berlin, Germany from March 18-22, 2019, where you can see some of these new features in action!

* In 3.7.0 this view is called "Card View" but due to a trademark clash it will be renamed to "Tile View" starting with 3.7.1

 

News date Thu, 12/06/2018 - 11:30
Categories: BI

Driving self-service analytics with SnapLogic and Workday Prism Analytics

SnapLogic - Wed, 12/05/2018 - 19:03

According to Gartner, “the analytics and business intelligence (BI) software market growth doubled to 11.5 percent in 2017, which is above the overall enterprise software market growth (about 10 percent).” Gartner also states that “modern BI platforms represent the highest-growing segment at 30 percent.” This growth indicates that organizations are keen on empowering their teams[...] Read the full article here.

The post Driving self-service analytics with SnapLogic and Workday Prism Analytics appeared first on SnapLogic.

Categories: ETL

Drupal's commitment to accessibility

Drupal - Wed, 12/05/2018 - 14:58

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

Last week, WordPress Tavern picked up my blog post about Drupal 8's upcoming Layout Builder.

While I'm grateful that WordPress Tavern covered Drupal's Layout Builder, it is not surprising that the majority of WordPress Tavern's blog post alludes to the potential challenges with accessibility. After all, Gutenberg's lack of accessibility has been a big topic of debate, and a point of frustration in the WordPress community.

I understand why organizations might be tempted to de-prioritize accessibility. Making a complex web application accessible can be a lot of work, and the pressure to ship early can be high.

In the past, I've been tempted to skip accessibility features myself. I believed that because accessibility features benefited a small group of people only, they could come in a follow-up release.

Today, I've come to believe that accessibility is not something you do for a small group of people. Accessibility is about promoting inclusion. When the product you use daily is accessible, it means that we all get to work with a greater number and a greater variety of colleagues. Accessibility benefits everyone.

As you can see in Drupal's Values and Principles, we are committed to building software that everyone can use. Accessibility should always be a priority. Making capabilities like the Layout Builder accessible is core to Drupal's DNA.

Drupal's Values and Principles translate into our development process, as what we call an accessibility gate, where we set a clearly defined "must-have bar." Prioritizing accessibility also means that we commit to trying to iteratively improve accessibility beyond that minimum over time.

Together with the accessibility maintainers, we jointly agreed that:

  1. Our first priority is WCAG 2.0 AA conformance. This means that in order to be released as a stable system, the Layout Builder must reach Level AA conformance with WCAG. Without WCAG 2.0 AA conformance, we won't release a stable version of Layout Builder.
  2. Our next priority is WCAG 2.1 AA conformance. We're thrilled at the greater inclusion provided by these new guidelines, and will strive to achieve as much of it as we can before release. Because these guidelines are still new (formally approved in June 2018), we won't hold up releasing the stable version of Layout Builder on them, but are committed to implementing them as quickly as we're able to, even if some of the items are after initial release.
  3. While WCAG AAA conformance is not something currently being pursued, there are aspects of AAA that we are discussing adopting in the future. For example, the new 2.1 AAA "Animations from Interactions", which can be framed as an achievable design constraint: anywhere an animation is used, we must ensure designs are understandable/operable for those who cannot or choose not to use animations.

Drupal's commitment to accessibility is one of the things that makes Drupal's upcoming Layout Builder special: it will not only bring tremendous and new capabilities to Drupal, it will also do so without excluding a large portion of current and potential users. We all benefit from that!

Categories: CMS

Unconference 2018

Liferay - Wed, 12/05/2018 - 09:50

//La versión española del artículo se puede encontrar aquí:  Unconference 2018.

Unconference took place on November 6, in Pakhuis de Zwijger, previously at Liferay DevCon in Amsterdam. I've read about this sessions, but I've never taken part in one. Spoiler, I loved it

If you have ever been taking part of one, you'll know Unconference agenda doesn't exist before it starts. If you haven't, I'm going to talk about Liferay Unconference 2018 to explain how it was.

First of all, Olaf talked about different spaces (talks zones, lunch zone, toilets, etc.), lunchtime, organization, and, very important, about 4 principles an 1 law.

4 principles:

  • Whoever comes is the right people (every participant wants to be there)
  • Whatever happens is the only thing that could have (the important thing is what it's happening in that place and in that moment )
  • Whenever it starts is the right time (you don't have to wait for anything to start the session)
  • When it’s (not) ever it’s (not) over (you make the most of the session)

1 law:

  • Law of Two Feet: If at any time you are in a situation where you are neither learning nor contributing, then you can use your two feet to go to a more productive place. This law is important to understand that every participant is present voluntarily.

I would like to remember a sentence that is important to understand Unconference mindset "If nobody attends your session, you have two options. In one hand, you can look for another interesting session,  in the other hand, you can work in your topic because when you propose a session, you are booking one hour to work in that topic".

At that moment we started to brainstorm and build out the day's agenda. We then write down topics on cards and choose the track.

Finally, these session topics are coordinated into available time slots and organized to avoid topic repetition and allow attendees to join the sessions they believe are right for them. It's important to know, only who has proposed the sessions, can change the card to another slot. So, if you want to change it (because there is another session at the same time), you must talk to her/him.

 

10

11

12

13

14

15

16

AStaging 2.0 - change listLiferay in action at the university

 

Rest/HATEOAS vs.GraphQLSennaJS/Module loader with other JS libsLiferay AnalyticsWhat extension point are you missing? BAPIO?Liferay performance with more than 1 million users Personalization - use cases & moreDDM, Forms & WorkflowSearch optimisation: boosting & filteringDXP Cloud C   How do you monitor your Liferay application & pluginsUsing multitenancy (more instances in one installtion) - a good idea?mongoDB & LiferayAudience Targeting to guest user: how to do it? D"Internal classes" - Why are some classes in the "internal" package not exported? How to customize?Container + Liferay: How to deploy or upgrade? Liferay intelliJ Plugin - Language Server ProtocolAdministration experiences GDPR EMigration experiences: LR 6.2 --> 7.0Liferay as a headless CMS - best practices Liferay + TensorFlowData integration, ETL/ESB | Experiences & methods integrating external systemsMobile with LiferayWorkspace: tips & tricks | How you use/extend workspace? Use plugins? FLiferay CommerceConfig & content orchestration - site initializers Your experience with Liferay + SSO DXP vs. CE  GReact?  Media/Video - server user upload/integrationDocumentation: What is missing?Making it easier for business users to build sites (modern site building)Liferay for frontend developers

H

       

I share some notes of the sessions:

  • Migration experiences : LR 6.2 --> 7.0           
    • If you want to upgrade Liferay version and to migrate to open source base data, it's better to migrate base data version first.
  • Config & content orchestration - site initializers            
    • There are some implementations, one is created by Commerce team.
    • Liferay team is analyzing to include it to new versions.
  • Liferay + TensorFlow
    • It is a new functionality in 7.2. Now it's available in github.
    • This functionality is implemented for images but is available to assets.  
  • How do you monitor your Liferay application & plugins
    • Use jvm to check threads           
    • Java thread dump analyzer -> http://fastthread.io
  • Documentation: What is missing?
    • Slack vs forum – don't lose information 
  • DXP vs. CE   
    • We talked about the two versions (CE and DXP) and possibility we pay licenses for support and code of both versions are the same
  • Workspace: tips & tricks | How you use/extend workspace? Use plugins?          
    • In the upgrade process to 7 version, if you can't migrate all modules, they recommend migrating services.
    • compile only - instruction call to proof

I think Unconference is very useful for everyone. On one hand, Liferay staff can check their functionalities and get feedback, and on the other hand, we can talk about topics important to us with experts.

To finish Unconference, we met in the plenary room and summarized the event.

Some points are very interesting of unconference are:

  • Philosophy/Mentality 
  • Attitude (everyone wants to share, teach and learn)
  • You can propose topics easily
  • You have a lot of options to learn (too many experts). 

As I said at the beginning, Unconference has been a wonderful experience and I would like to come back again.

Álvaro Saugar 2018-12-05T14:50:00Z
Categories: CMS, ECM
Syndicate content