Support cluster-wide HTTP Proxy configuration in IBM StreamSets Cartridge.

This is the current workaround for this issue:

- IBM Streamsets Cartridge is currently not listed as supported component for Proxy Configuration
(see https://www.ibm.com/docs/en/software-hub/5.2.x?topic=environment-applying-cluster-http-proxy-settings)
- The workaround below will enable IBM Streamsets to work smoothless in a proxy enabled environment

##### Contents
1. Install IBM Streamsets Cartridge
   1.1. update proxy config with StreamSets services
   1.2. update no_proxy list to remove double comma ",," 
   1.3. update proxy allow_list
2. Configure Agent
3. Configure Deployment/Engine

##### 1. Install IBM Streamsets Cartridge
##### 1.1. update proxy config with StreamSets services
----- PROBLEM -----
Following the standard documentation to install here: https://www.ibm.com/docs/en/software-hub/5.2.x?topic=streamsets-installing
The installation will get stuck during step "3. Create the custom resource for IBM StreamSets."
   - a detailed investigation regarding cr status will show progress around 60%
   - a detailed investigation in the ibm-streamsets-* pods will show multiple HTTP errors / Proxy errors

----- ROOT CAUSE -----
* During installation the different ibm-streamsets-* pods will communicate with each other.
* Since the new internal services created during StreamSets installation are not yet included in the no_proxy list, the pods will try to communicate via proxy. This communication fails

----- SOLUTION -----
* Monitor streamsets cr status and check for progress. Wait for the progress to reach 60% and not progressing further
* verify that the streamsets services are created 
   OpenShift Console --> Workloads (namespace "cpd" or similar) -> Services
   filter for ibm-streamsets
   Result should include services like:
       ibm-streamsets-aster-security-ui-v1,
       ibm-streamsets-astersecurity,
       ibm-streamsets-banner,
       ibm-streamsets-connection,
       ibm-streamsets-docs,
       ibm-streamsets-jobrunner,
       ibm-streamsets-messaging,
       ibm-streamsets-metering,
       ibm-streamsets-mysql-deploy,
       ibm-streamsets-mysql-deploy-headless,
       ibm-streamsets-notification,
       ibm-streamsets-pipelinestore,
       ibm-streamsets-provisioning,
       ibm-streamsets-scheduler,
       ibm-streamsets-security,
       ibm-streamsets-sequencing,
       ibm-streamsets-sla,
       ibm-streamsets-static-content,
       ibm-streamsets-timeseries,
       ibm-streamsets-topology,
       ibm-streamsets-tunneling
* apply proxy configuration as documented (see https://www.ibm.com/docs/en/software-hub/5.2.x?topic=environment-applying-cluster-http-proxy-settings)
   --> this will include the new ibm-streamsets services in the no_proxy list.
* delete cr as documented 
   --> see https://www.ibm.com/docs/en/software-hub/5.2.x?topic=streamsets-uninstalling
   --> step 2. Delete the custom resource for IBM StreamSets)
* resume installation following the install guide 
   --> https://www.ibm.com/docs/en/software-hub/5.2.x?topic=streamsets-installing
   --> step "3. Create the custom resource for IBM StreamSets."

##### 1.2. update no_proxy list to remove double comma ",,"
----- PROBLEM -----
during installation the ibm-streamsets-provisioning pod throws an error that is stopping the provisioning pod 
   -->String Index Out Of Range exception when processing the non-proxy hosts.

----- ROOT CAUSE -----
The no_proxy list is autogenerated by the cpd-cli manage create-proxy-config CLI
(see here: https://www.ibm.com/docs/en/software-hub/5.2.x?topic=environment-applying-cluster-http-proxy-settings)
this process includes a double comma (",,")
   --> "cpd.svc.cluster.local,,aiopenscale-ibm-aios-bias-svc"
   --> It's likely that trying to process the empty entry in the list is causing the String Index Out of Range error.

(Note: this issue is already addressed to IBM Software Hub development, might be resolved in future releases)

----- SOLUTION -----
Manually correct the no_proxy list and remove the double comma
1. cpd-cli manage create-proxy-config 
   (with parameters as described in the above documentation)
2. edit ./cpd-cli-workspace/olm-utils-workspace/work/rsi/proxy-envs.json in the cpd-cli directory
3. apply changed values to the rsi patch: 
   cpd-cli manage create-rsi-patch --cpd_instance_ns=cpd --patch_name=proxy-patch --patch_spec=/tmp/work/rsi/proxy-envs.json 
4. verify that changs are saved correctly:
   cpd-cli manage get-rsi-patch-info --cpd_instance_ns=cpd --all
5. apply proxy patch with 
   cpd-cli manage enable-proxy --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS}
   (as in above documentation)
6. afterwards delete "ibm-streamsets-provisioning" pods (they will be recreated automatically now with the correct env values)

----- RESULT -----
the provisioning pod completed successfully and streamsets install was completed. 
the default streamsets control hub instance is created
the landing page can be accessed

#####     1.3. update proxy allow_list
----- PROBLEM -----
creating engine versions from public repository (archives.streamsets.com) fails, since the URL is not allowed by proxy

----- SOLUTION -----
update the proxy allow list with the follwoing hosts:
   "archives.streamsets.com",
    "accounts.streamsets.com"

##### 2. Configure Agent
see https://www.ibm.com/docs/en/streamsets-controlhub?topic=environments-kubernetes#task_tgv_rrg_lwb
pay attention to the no_proxy parameter:
   depending on technology, the wildcard filters work differently.
   in Java you have to use "*" as wildcard charater. the "." notation like ".my.domain" does not work with Java
   Be sure to include at least the cluster ip 
       --> If a proxy is between agent and control hub, you also have to extend the proxy allow list with the cpd-url
       --> if the agent and control hub run on the same openshift cluster, include the cpd-url in the no_proxy lsit as well
           e.g., -Dhttp.nonProxyHosts=<cluster-IP>|<cpd url>

##### 3. Configure Deployment/Engine
see https://www.ibm.com/docs/en/streamsets-controlhub?topic=environments-kubernetes#task_a2m_gnm_lwb
pay attention to the "jdk.http.auth.tunneling.disabledSchemes=" parameter. By default "Basic" authentication is disabled.
If you still get proxy authentication errors in the logs, try leaving this parameter empty.
see also https://www.ibm.com/docs/en/streamsets/6.x?topic=configuration-using-proxy-server for detailed description of the individual paramters