Skip to content

Issue 144 remove named objects #422

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Aug 22, 2019
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 32 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Many organizations are using WebLogic Server, with or without other Oracle Fusio
- [Simple Example](#simple-example)
- [Model Names](#model-names)
- [Model Semantics](#model-semantics)
- [Declaring Named MBeans to Delete](#declaring-named-mbeans-to-delete)
- [Administration Server Configuration](site/admin_server.md)
- [Model Security](site/security.md)
- [Modeling Security Providers](site/security_providers.md)
Expand Down Expand Up @@ -52,7 +53,7 @@ As new use cases are discovered, new tools will likely be added to cover those o

## The Metadata Model

The metadata model (or model, for short) is a version-independent description of a WebLogic Server domain configuration. The tools are designed to support a sparse model so that the model need only describe what is required for the specific operation without describing other artifacts. For example, to deploy an application that depends on a JDBC data source into an existing domain that may contain other applications or data sources, the model needs to describe only the application and the data source in question. If the datasource was previously created, the `deployApps` tool will not try to recreate it but may update part of that data source's configuration if the model description is different than the existing values. If the application was previously deployed, the `deployApps` tool will compare the binaries to determine if the application needs to be redeployed or not. In short, the `deployApps` tool supports an iterative deployment model so there is no need to change the model to remove pieces that were created in a previous deployment.
The metadata model (or model, for short) is a version-independent description of a WebLogic Server domain configuration. The tools are designed to support a sparse model so that the model need only describe what is required for the specific operation without describing other artifacts. For example, to deploy an application that depends on a JDBC data source into an existing domain that may contain other applications or data sources, the model needs to describe only the application and the data source in question. If the data source was previously created, the `deployApps` tool will not try to recreate it but may update part of that data source's configuration if the model description is different than the existing values. If the application was previously deployed, the `deployApps` tool will compare the binaries to determine if the application needs to be redeployed or not. In short, the `deployApps` tool supports an iterative deployment model so there is no need to change the model to remove pieces that were created in a previous deployment.

The model structure, and its folder and attribute names, are based on the WLST 12.2.1.3 offline structure and names with redundant folders removed to keep the model simple. For example, the WLST path to the URL for a JDBC data source is `/JDBCSystemResource/<data-source-name>/JdbcResource/<data-source-name>/JDBCDriverParams/NO_NAME_0/URL`. In the model, it is `resources:/JDBCSystemResource/<data-source-name>/JdbcResource/JDBCDriverParams/URL` (where `resources` is the top-level model section where all WebLogic Server resources/services are described).

Expand Down Expand Up @@ -220,6 +221,36 @@ In the example above, the `Target` attribute is specified three different ways,

One of the primary goals of the WebLogic Deploy Tooling is to support a sparse model where the user can specify just the configuration needed for a particular situation. What this implies varies somewhat between the tools but, in general, this implies that the tools are using an additive model. That is, the tools add to what is already there in the existing domain or domain templates (when creating a new domain) rather than making the domain conform exactly to the specified model. Where it makes sense, a similar, additive approach is taken when setting the value of multi-valued attributes. For example, if the model specified the cluster `mycluster` as the target for an artifact, the tooling will add `mycluster` to any existing list of targets for the artifact. While the development team has tried to mark attributes that do not make sense to merge accordingly in our knowledge base, this behavior can be disabled on an attribute-by-attribute basis, by adding an additional annotation in the knowledge base data files. The development team is already thinking about how to handle situations that require a non-additive, converge-to-the-model approach, and how that might be supported, but this still remains a wish list item. Users with these requirements should raise an issue for this support.

### Declaring Named MBeans to Delete

With WebLogic Deploy Tooling release 1.3.0, you can specify named items in the model to be deleted using the Create Domain, Update Domain, and Deploy Applications Tools. Named items are those that have multiple instances that are distinguished by user-provided names, such as managed servers, data sources, and security realms. Items to be deleted are prepended with an exclamation point (!) in the model.

In this example, the managed server ```obsoleteServer``` will be deleted, and ```newServer``` will be created:

```yaml
Server:
!obsoleteServer:
newServer:
ListenAddress: 127.0.0.1
ListenPort: 9005
```

This feature can also remove items that were created by WebLogic Server templates. For example, the basic template creates a default security realm called ```myrealm```. If a user chooses to declare a custom realm, ```myrealm``` is no longer needed. In this example, ```myrealm``` will be deleted, and the custom realm ```newrealm``` will be created, and declared as the default realm:

```yaml
SecurityConfiguration:
DefaultRealm: newrealm
Realm:
!myrealm:
newrealm:
AuthenticationProvider:
...
```

This feature does not apply to named security providers within a realm. These items follow a special set of rules that are required to maintain their ordering. See [Modeling Security Providers](site/security_providers.md) for detailed information.

This feature cannot be use to un-deploy applications or remove libraries.

### Using Multiple Models

The Create Domain, Update Domain, Deploy Applications, and Validate Model Tools allow the specification of multiple models on the command line. For example:
Expand Down
1 change: 1 addition & 0 deletions core/src/main/antlr4/oracle/weblogic/deploy/yaml/Yaml.g4
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,7 @@ fragment ID_START
: [_]
| [A-Z]
| [a-z]
| '!'
;

fragment ID_CONTINUE
Expand Down
3 changes: 3 additions & 0 deletions core/src/main/python/wlsdeploy/tool/create/creator.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,9 @@ def _create_named_mbeans(self, type_name, model_nodes, base_location, log_create
existing_folder_names = self._get_existing_folders(list_path)
for model_name in model_nodes:
name = self.wlst_helper.get_quoted_name_for_wlst(model_name)
if deployer_utils.is_delete_name(name):
deployer_utils.delete_named_element(location, name, existing_folder_names, self.alias_helper)
continue

if token_name is not None:
location.add_name_token(token_name, name)
Expand Down
4 changes: 4 additions & 0 deletions core/src/main/python/wlsdeploy/tool/deploy/deployer.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,10 @@ def _add_named_elements(self, type_name, model_nodes, location):

token = self.alias_helper.get_name_token(location)
for name in model_nodes:
if deployer_utils.is_delete_name(name):
deployer_utils.delete_named_element(location, name, existing_names, self.alias_helper)
continue

is_add = name not in existing_names
log_helper.log_updating_named_folder(type_name, name, parent_type, parent_name, is_add, self._class_name,
_method_name)
Expand Down
48 changes: 48 additions & 0 deletions core/src/main/python/wlsdeploy/tool/deploy/deployer_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,54 @@ def merge_lists(existing_list, model_list, separator=',', return_as_string=True)
return result


def is_delete_name(name):
"""
Determines if the specified name is flagged for deletion with the "!" prefix.
:param name: the name to be checked
:return: True if the name is prefixed, false otherwise
"""
return name.startswith("!")


def get_delete_item_name(name):
"""
Returns the WLST name of the item to be deleted.
Removes the "!" prefix from the name. An exception is thrown if the name is not prefixed.
:param name: the prefixed model name of the item to be deleted
:return: the model name of the item to be deleted
"""
_method_name = 'get_delete_item_name'

if is_delete_name(name):
return name[1:]

ex = exception_helper.create_deploy_exception('WLSDPLY-09111', name)
_logger.throwing(ex, class_name=_class_name, method_name=_method_name)
raise ex


def delete_named_element(location, delete_name, existing_names, alias_helper):
"""
Delete the specified named element if present. If the name is not present, log a warning and return.
:param location: the location of the element to be deleted
:param delete_name: the name of the element to be deleted, assumed to include the "!" prefix
:param existing_names: a list of names to check against
:param alias_helper: alias helper for lookups
"""
_method_name = 'delete_named_element'

name = get_delete_item_name(delete_name)
type_name = alias_helper.get_wlst_mbean_type(location)

if name not in existing_names:
_logger.warning('WLSDPLY-09109', type_name, name, class_name=_class_name, method_name=_method_name)
else:
_logger.info('WLSDPLY-09110', type_name, name, class_name=_class_name, method_name=_method_name)
type_path = alias_helper.get_wlst_create_path(location)
_wlst_helper.cd(type_path)
_wlst_helper.delete(name, type_name)


def ensure_no_uncommitted_changes_or_edit_sessions():
"""
Ensure that the domain does not contain any uncommitted changes and there is no existing edit session.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,9 @@ def _add_jndi_properties(self, property_name_nodes, location):
if len(property_name_nodes) == 0:
return

# use a copy of the dictionary to remove items as they are deleted
remaining_name_nodes = property_name_nodes.copy()

parent_type, parent_name = self.get_location_type_and_name(location)
is_online = self.wlst_mode == WlstModes.ONLINE
if is_online and deployer_utils.is_in_resource_group_or_template(location):
Expand All @@ -151,10 +154,16 @@ def _add_jndi_properties(self, property_name_nodes, location):
name_attribute = self.alias_helper.get_wlst_attribute_name(properties_location, KEY)
mbean_type = self.alias_helper.get_wlst_mbean_type(properties_location)

for property_name in property_name_nodes:
if deployer_utils.is_delete_name(property_name):
name = deployer_utils.get_delete_item_name(property_name)
self._delete_mapped_mbean(properties_location, properties_token, mbean_type, name_attribute, name)
del remaining_name_nodes[property_name]

# loop once to create and name any missing folders.
folder_map = self._build_folder_map(properties_location, properties_token, name_attribute)

for property_name in property_name_nodes:
for property_name in remaining_name_nodes:
folder_name = dictionary_utils.get_element(folder_map, property_name)
if folder_name is None:
self.wlst_helper.cd(foreign_server_path)
Expand All @@ -164,7 +173,7 @@ def _add_jndi_properties(self, property_name_nodes, location):
# loop a second time to set attributes
new_folder_map = self._build_folder_map(properties_location, properties_token, name_attribute)

for property_name in property_name_nodes:
for property_name in remaining_name_nodes:
is_add = property_name not in folder_map
log_helper.log_updating_named_folder(JNDI_PROPERTY, property_name, parent_type, parent_name, is_add,
self._class_name, _method_name)
Expand All @@ -173,7 +182,7 @@ def _add_jndi_properties(self, property_name_nodes, location):
properties_location.add_name_token(properties_token, folder_name)
self.wlst_helper.cd(self.alias_helper.get_wlst_attributes_path(properties_location))

property_nodes = property_name_nodes[property_name]
property_nodes = remaining_name_nodes[property_name]
self.set_attributes(properties_location, property_nodes)

def _add_group_params(self, group_name_nodes, location):
Expand All @@ -188,6 +197,9 @@ def _add_group_params(self, group_name_nodes, location):
if len(group_name_nodes) == 0:
return

# use a copy of the dictionary to remove items as they are deleted
remaining_name_nodes = group_name_nodes.copy()

parent_type, parent_name = self.get_location_type_and_name(location)
template_path = self.alias_helper.get_wlst_subfolders_path(location)
groups_location = LocationContext(location)
Expand All @@ -196,11 +208,20 @@ def _add_group_params(self, group_name_nodes, location):
name_attribute = self.alias_helper.get_wlst_attribute_name(groups_location, SUB_DEPLOYMENT_NAME)
mbean_type = self.alias_helper.get_wlst_mbean_type(groups_location)

for group_name in group_name_nodes:
if deployer_utils.is_delete_name(group_name):
group_nodes = group_name_nodes[group_name]
name = deployer_utils.get_delete_item_name(group_name)
sub_name = self._get_subdeployment_name(group_nodes, name)
self._delete_mapped_mbean(groups_location, groups_token, mbean_type, name_attribute, sub_name)
del remaining_name_nodes[group_name]

# loop once to create and name any missing folders.
folder_map = self._build_folder_map(groups_location, groups_token, name_attribute)

for group_name in group_name_nodes:
sub_deployment_name = self._get_subdeployment_name(group_name_nodes, group_name)
for group_name in remaining_name_nodes:
group_nodes = remaining_name_nodes[group_name]
sub_deployment_name = self._get_subdeployment_name(group_nodes, group_name)
folder_name = dictionary_utils.get_element(folder_map, sub_deployment_name)
if folder_name is None:
self.wlst_helper.cd(template_path)
Expand All @@ -210,19 +231,52 @@ def _add_group_params(self, group_name_nodes, location):
# loop a second time to set attributes
new_folder_map = self._build_folder_map(groups_location, groups_token, name_attribute)

for group_name in group_name_nodes:
sub_deployment_name = self._get_subdeployment_name(group_name_nodes, group_name)
for group_name in remaining_name_nodes:
group_nodes = remaining_name_nodes[group_name]
sub_deployment_name = self._get_subdeployment_name(group_nodes, group_name)
is_add = sub_deployment_name not in folder_map
log_helper.log_updating_named_folder(GROUP_PARAMS, group_name, parent_type, parent_name, is_add,
self._class_name, _method_name)

folder_name = dictionary_utils.get_element(new_folder_map, sub_deployment_name)
groups_location.add_name_token(groups_token, folder_name)
self.wlst_helper.cd(self.alias_helper.get_wlst_attributes_path(groups_location))

group_nodes = group_name_nodes[group_name]
self.set_attributes(groups_location, group_nodes)

def _delete_mapped_mbean(self, folder_location, folder_token, mbean_type, name_attribute, name):
"""
Delete the child MBean with an attribute value that matches the specified name.
This requires looping through the child MBeans for each deletion, since the MBean names change on delete.
:param folder_location: the WLST location for the folder with named sub-elements
:param folder_token: the folder token used to iterate through the MBean names
:param mbean_type: the MBean type of the sub-elements
:param name_attribute: the attribute for the name in each sub-element
:param name: the name of the sub-element to be deleted
"""
_method_name = '_delete_mapped_mbean'

original_path = self.wlst_helper.get_pwd()
mapped_folder_name = None

folder_names = deployer_utils.get_existing_object_list(folder_location, self.alias_helper)
for folder_name in folder_names:
folder_location.add_name_token(folder_token, folder_name)
self.wlst_helper.cd(self.alias_helper.get_wlst_attributes_path(folder_location))
attribute_value = self.wlst_helper.get(name_attribute)
if attribute_value == name:
mapped_folder_name = folder_name
break

self.wlst_helper.cd(original_path)

if mapped_folder_name is None:
self.logger.warning('WLSDPLY-09109', mbean_type, name,
class_name=self._class_name, method_name=_method_name)
else:
self.logger.info('WLSDPLY-09110', mbean_type, name,
class_name=self._class_name, method_name=_method_name)
self.wlst_helper.delete(mapped_folder_name, mbean_type)

def _build_folder_map(self, folder_location, folder_token, name_attribute):
"""
Build a map of existing sub-element names to folders.
Expand All @@ -245,15 +299,14 @@ def _build_folder_map(self, folder_location, folder_token, name_attribute):
self.wlst_helper.cd(original_path)
return folder_map

def _get_subdeployment_name(self, group_name_nodes, group_name):
def _get_subdeployment_name(self, group_nodes, group_name):
"""
Returns the derived sub-deployment name for the specified group name.
The sub-deployment name attribute is returned if specified, otherwise the group name is returned.
:param group_name_nodes: the group name nodes from the model
:param group_nodes: the group nodes from the model
:param group_name: the group name being examined
:return: the derived sub-deployment name
"""
group_nodes = group_name_nodes[group_name]
sub_deployment_name = dictionary_utils.get_element(group_nodes, SUB_DEPLOYMENT_NAME)
if sub_deployment_name is not None:
return sub_deployment_name
Expand Down
4 changes: 4 additions & 0 deletions core/src/main/python/wlsdeploy/tool/util/topology_helper.py
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,10 @@ def create_placeholder_named_elements(self, location, model_type, model_nodes):

name_nodes = dictionary_utils.get_dictionary_element(model_nodes, model_type)
for name in name_nodes:
if deployer_utils.is_delete_name(name):
# don't create placeholder for delete names
continue

if name not in existing_names:
self.logger.info('WLSDPLY-19403', model_type, name, class_name=self.__class_name,
method_name=_method_name)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -936,6 +936,9 @@ WLSDPLY-09106=Shared library name {0} contained {1} @ signs when only zero or on
WLSDPLY-09107=Shared library name {0} contained {1} # signs when only zero or one are allowed
WLSDPLY-09108=Model attribute {0} at model location {1} with value {2} references a location inside \
the archive file but the archive file was not provided
WLSDPLY-09109=Unable to delete {0} {1}, name does not exist
WLSDPLY-09110=Deleting {0} {1}
WLSDPLY-09111=Unable to get delete item name for {0}

# wlsdeploy/tool/deploy/deployer.py
WLSDPLY-09200=Error setting attribute {0} for {1} {2}: {3}
Expand Down