Combine Raster Bands

To combine multiple single-band rasters into a single multi-band raster, use the RasterBandCombiner transformer.
For example, combine the DTM and DSM height rasters into a single 2-band raster. Or combine the pluvial, fluvial and combined depth rasters into a single 3-band raster.

See also https://blog.safe.com/2018/08/bring-2d-data-cesiumjs/

Overlapping raster with vector data

When overlaying rasters with lines or polygons, don't convert the raster to vector format, but instead convert the line or polygon dataset to raster.

First use a 3DForcer transformer and set the 'elevation' of your lines or polygons to 1. Then use the NumericRasterizer transformer, setting 0 as the Background Value. This will create a binary raster where the lines or polygons have a raster value of 1, and everything else 0.

Then point this rasterized vector dataset and the original raster both to the RasterExpressionEvaluator transformer, and do Preserve; A[0]*[B0]. This will effectively give an intersection raster.

Remove duplicates

To remove duplicate records, use the DuplicateFilter transformer.

To remove duplicate geometries, use the Matcher transformer, and take the output from the SingleMatched and NotMatched ports:

ArcPy in FME

To enable ArcPy in FME:

1. Make sure you have ArcGIS Desktop Background Geoprocessing (64-bit) and the 64-bit version of FME installed;
2. In FME, go to Tools > FME Options... > Translation;
3. Set the Preferred Python Interpreter to C:\Windows\System32\Python32.dll.

After that you can just type in import arcpy in your Python code:

See also https://knowledge.safe.com/articles/814/choosing-a-different-python-interpreter-installati.html and https://knowledge.safe.com/articles/29799/importing-arcpy-troubleshooting-ideas.html

(Note: this solution may work for me because I have ArcGIS Pro installed.)

The Effect of Spatial Transformer Choice on Performance

Spatial analysis can be processing-intensive, particularly when a large number of features are involved. If you would like to tune the performance of your workspace, this is a good place to start.

When there are multiple ways to configure a workspace to reach the same goal, it is often best to choose the transformer most specifically suited to your task. As an example, when comparing address points to building polygons, there are a few ways to approach it.

The first example, using a SpatialFilter to test whether or not points fall inside polygons, produces the correct result. But the SpatialFilter is a fairly complex transformer, able to test for multiple conditions and accept a wide range of geometry types. It isn’t optimized for the specific spatial relationship we are analyzing here.

With a SpatialFilter:

The second example uses a PointOnAreaOverlayer, followed by a Tester. The features output are the same as in the first method, but the transformer is optimized for this specific task. The difference in processing time is substantial - from 54.3 seconds in the first configuration, down to 13.7 seconds in the second one.

With a PointOnAreaOverlayer and a Tester:


If performance is an issue in your workspace, look for alternative methods, guided by geometry.

Source: https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/topologybuilder.htm

StringSearcher

To split a string into its components using Regular Expressions, you can use the StringSearcher transformer. In the [Contains Regular Expression] field, put parentheses around each group; and in the [Subexpression Matches List Name] field within the [Advanced] group, specify the name of the list that will contain the individual components.

See also https://www.safe.com/webinars/back-fme-school-day-1-data-fme/ at around 1:50:00

FME Includer

1. Create a User Parameter of type Choice with options Yes/No asking whether to include X:





2. Set up a Tester transformer, with the Pass Criteria set to OR. In the first Test Clause, specify all records that are always to go through. In the second Test Clause, specify $(IncludeX) = Yes.

If the user selects [No], i.e. don’t include X, the second test clause is always False, so only the records that comply with the first test clause go through.
If the user selects [Yes], the second test clause is always True, so all records go through, i.e. no filtering takes place.

Create New Geometry

When creating new geometry (e.g. VertexCreator), use the CoordinateSystemSetter transformer to define the coordinate system.

---

To change the geometry of a line (or polygon) into a single point at its start or end, you can use the Snipper transformer, setting the [Snipping Mode] to [Vertex], and setting the two indexes to [0] for the first vertex, or to [-1] for the last vertex:

---

To create a line or polygon from an array of coordinates, you can use the VertexCreator transformer with the [Mode] set to [Add Point]. However, this doesn't work if there are duplicate points other than start = end in the array.

Instead use the VertexCreator transformer with the [Mode] set to [Replace with Point], followed by a LineBuilder transformer. On the latter, make sure you connect the [Line] or [Polygon] output port!

Get X & Y

To get the X and Y coordinates as attributes, you can use the CoordinateExtractor transformer, using index 0 for the first vertex and index -1 for the last.

Raster Checking

Before processing a raster, check the raster. After loading the raster in the Data Inspector, select the Information button and drag a box to show the raster details.

Clicking in the raster, shows the raster cell details of the clicked cell.

Raster Bands

If the raster is a colour raster, i.e. an image, it should have 3 bands displaying the RGB values;
If the raster is a numerical raster, e.g. a DTM, it should have 1 band displaying the numerical value.

To remove any unwanted bands (e.g. alpha band), use the RasterSelector transformer followed by the RasterBandKeeper or RasterBandRemover transformer.

To separate the RGB bands, use the RasterBandSeparator transformer.

Raster NoData

NoData by default means no colour = transparent.

To set the raster's NoData value on a numerical raster, first define the NoData value using the RasterBandNodataSetter, followed by a RasterCellValueReplacer setting all values outside the valid domain to the newly selected NoData value.

Raster Display

To display a raster, use the RasterInterpretationCoercer transformer.

Python: Get Filename and Folder Name

Filename

# Get the full name of the input DWG file
s1 = FME_MacroValues['InputDWG']

# If there is a backslash, take the bit after the last one
if "\\" in s1:
    s2 = s1.split('\\')[-1]

# if the output file is in the same folder as the workbench,
# FME uses a relative folder without slashes: $(FME_MF_DIR)foo.bar
elif "FME_MF_DIR" in s1:
    s2 = s1[13:]

# Chop off the extension
return s2.split('.')[0]

UpdateDetector

The UpdateDetector custom transformer compares two datasets and specifies which records have been inserted, deleted and updated.

The fme_db_operation output attribute can be used on the Feature Operation setting on the (database?) Writer. See https://www.safe.com/webinars/back-fme-school-day-1-data-fme/ at 1h17m45s

Regular Expressions (Regex)

(Note to self: use Word to update the tables)

Metacharacters

Metacharacters are reserved characters, and they are:

{}[]()^$.|*+?-\

To find a metacharacter in a string, prefix it with a backslash, e.g. \. to find a dot, or \\ to find a backslash.

Find a character

a		a single character a
[abc]	(a|b|c)	any single character from the set, i.e. a, b, or c. For example, [bcr]at matches "bat", "cat", and "rat"
[^abc]		any single character except a, b, or c. For example, [^b]at matches "cat" and "rat", but not "bat"
[A-Z]		any character in the range A-Z, i.e. any uppercase letter
[a-z]		any character in the range a-z, i.e. any lowercase letter
[A-Za-z]		any character in the range A-Z or a-z, i.e. any letter
[0-9]	\d	any character in the range 0-9, i.e. any number
.		any single character
\.		a dot (prefix with a backslash as . is a metacharacter)
\\		a backslash character (prefix with a backslash as the backslash is a metacharacter itself)
\t		a tab character
\n		a new line character (also see $ is the end of line expression)
\s		any whitespace character, which includes space, tab and newline character
\w	[0-9a-zA-Z_]	any word character, which only includes any lowercase or uppercase letter, any number and the underscore character
\d		any digit
(a|b)		a or b

Line Characters

^	start of line
$	end of line
^$	empty string = ""

Quantification

a?	0 or 1 of a. For example, colou?r matches both "color" and "colour"
a*	0 or more of a. For example, ab*c matches "ac", "abc", "abbc", "abbbc", and so on
a+	1 or more of a. For example, ab+c matches "abc", "abbc", "abbbc", and so on, but not "ac"
a{3}	exactly 3 of a
a{3,}	3 or more of a
a{3,6}	between 3 and 6 of a

Grouping

(   )	capture group
\1	backreference to group #1

See also https://www.regular-expressions.info/index.html

Finding and replacing text

• To find substrings, use the StringSearcher transformer or the @FindString or @FindRegEx built-in functions
• To replace substrings, use the StringReplacer transformer or the @ReplaceString or@ReplaceRegEx built-in functions

DateTimeParse: String to Date/Time

To convert a String attribute containing a date or time text to a Date/Time attribute, use the @DateTimeParse built-in function.

To automatically adjust for number overflow (e.g. 32-Jan), always use 'repair' as its third argument, e.g.:

@DateTimeParse(@Right(@Value(fme_basename),16),%H_%M %d_%m_%Y,repair)