Skip to content

ReleaseNotes

shawnlaffan edited this page Feb 6, 2024 · 150 revisions

Table of contents:

Version 4.99 dev series

This is a development release series, leading towards version 5.0.

For the full list of issues and changes leading to the 5.0 release, see https://github.com/shawnlaffan/biodiverse/milestone/18

Main changes:

  • Data import
    • Spreadsheet imports from XLSX files are now substantially faster. Details are in Issue 882.
    • Shapefile imports are now slightly faster. Details are in Issue 885.
  • Indices:
    • The CANAPE super class is now calculated and plotted. Issue 890 Blog post
    • Indices for the Hurlbert species richness estimator are now available. The set of sample numbers may be changed or made more flexible in the future. Implementation is under issue 871.
    • The phylogenetic and range weighted turnover indices are now faster, particularly when used in pairwise mode such as when building a cluster matrix.
  • GUI
    • A basedata can be trimmed using another basedata (previously only matrices and trees could be used). The basedata trimming interface has been streamlined as part of this process. Issue 880. Blog post
    • The tree being displayed in any of the visualisation tabs can now be exported. Issue 902.
    • Trees can be plotted by depth across all visualisation tabs. Issue 906.
    • The tree plot control menu options are now a separate submenu. Issue 905. Blog post
    • Better colour schemes are now used by default when plotting indices that are zero-centred divergent and ratio based. Issue 820 Blog post pending.
    • Spatial tab
      • Tree branches can be coloured using list indices from any spatial output in the project. Issue 833. Blog post

Version 4.3

This release contains a small number of bug fixes and improved functionality. For the full list of issues and changes leading to the 4.3 release, see https://github.com/shawnlaffan/biodiverse/milestone/21

Main changes:

  • GUI:
    • z-score plotting has been fixed (colours were reversed). Issue 857.
  • Randomisations
  • Spatial conditions
    • The sp_points_in_same_poly_shape condition is now faster when any points do not intersect any polygons. See commit 3ca2703.

Version 4.2

This release contains a small number of bug fixes and improved functionality. For the full list of issues and changes leading to the 4.2 release, see https://github.com/shawnlaffan/biodiverse/milestone/20

Main changes:

  • GUI
    • Branch highlighting in the View Labels tab works again. This was broken in version 4.1. Issue #850.
  • Data imports
    • Raster imports now include the band labels if defined in multiband files. Issue #852.
    • Importing a raster now works when the nodata value is NaN. Issue #851.

Version 4.1

For the full list of issues and changes leading to the 4.1 release, see https://github.com/shawnlaffan/biodiverse/milestone/19

Main changes:

  • GUI
    • Z-score indices are now plotted using a five class divergent colour scheme to more clearly indicate which locations are in the outer 10%, 5% and 1% of the expected distribution. Issue 841. See also this blog post
  • Indices
    • A bug in the NRI, NTI and Net VPD indices meant undefined results were obtained the second time a tree was used, and when only one of NRI/NTI or VPD were calculated the first time. Issue 842.
    • The phylogenetic endemism indices have been further optimised. (There is no associated issue).
  • Imports
    • An issue with raster imports being offset has been fixed. Issue 845

Version 4.0

For the full list of issues and changes leading to the 4.0 release, see https://github.com/shawnlaffan/biodiverse/milestone/17

A series of blog posts goes into more detail about several of the changes: http://biodiverse-analysis-software.blogspot.com/search/label/Version4

  • General
    • As of 3.99_005 the Windows executable files are digitally signed. This will avoid OS warnings when files are downloaded.
  • Analyses
    • CANAPE
    • Cluster analyses
      • Cluster analyses will now lump all zero distance matrix pairs in an initial pass where possible. This avoids needless tie-breaker calculations when data sets have large sets of groups with the same label assemblages. Issue 760
      • Cluster and Region Grower analyses are no longer rebuilt by default under the randomisations, speeding up any randomisations that include them. Comparison of per-node calculations is still done, though, as this does not use the rebuilt tree. There is a user visible option in the GUI to enable the previous behaviour if it is needed. Issue 765
      • Cluster and region grower analyses now warn and provide an informative error message when there is only a single group in the basedata. Issue 782
    • Randomisations
  • Calculations
    • A local PD calculation, "Phylogenetic Diversity (local)", now allows the calculation of PD and PD_P to the last common ancestor, as opposed to the root node. Issue 762
    • A new calculation, "Last shared ancestor properties", provides details about the last shared ancestor for a set of labels, for example the length from its tipwards end to the root node. Issue 763
    • MPD, MNTD, NRI & NTI: The MPD and MNTD calculations are now substantially faster. The NRI and, for ultrametric trees, NTI calculations are orders of magnitude faster as they use expected values calculated using the exact methods described in Tsirogiannis et al. (2012) and Tsiriogiannis et al. (2014). There is also no longer a performance penalty under the randomisations for outputs containing NRI and NTI indices when richness scores are held constant (which is the default). This is because the expected values for a given tree are re-used across analyses. Issue 786, Issue 789, Issue 790. More details are in the blog post.
    • The taxonomic distinctness calculations have been moved to the BiodiverseX namespace and so are not available in the GUI by default. Issue 751.
    • The calculation of median and percentile values for the element properties have changed. Issue #798. More details in the blog post.
    • The phylogenetic distincness variance (VPD) is now calculated with the mean and other related statistics. The net VPD and its component indoces can also be calculated to assess significance of the net VPD against random resampling. Issue #811 See blog post
  • Spatial Conditions
    • New condition sp_richness_greater_than() allows the neighbour set to be defined as the set of groups with a richness exceeding a threshold. Issue 783.
    • New condition sp_redundancy_greater_than() allows the neighbour set to be defined as the set of groups with a sample redundancy exceeding a threshold.
    • New conditions sp_point_in_cluster() and sp_points_in_same_cluster() that can be used to model polygons from clusters in a Cluster or RegionGrower analysis without first needing to export them to a shapefile and any subsequent processing to extract the relevant parts. Issue 803. More details in the blog post.
  • Imports
    • Group properties can now be directly imported from rasters. This makes it much easier to analyse environmental patterns in relation to taxonomic and/or phylogenetic ones. Issue 761. More details are in the blog post
  • Exports
    • Cluster and RegionGrower analyses can be exported to shapefile format in a grouped form. This models the cluster display where multiple sub-clusters are coloured. The output file is somewhat awkward and needs further processing, which is why the sp_points_in_same_cluster() spatial condition was developed (see above). Issue 757. More details are in the blog post
  • Trees
    • A new option has been added to merge only-child ("knuckle") nodes with their parents. This is probably most useful after trimming a tree but can be applied to any tree. Issue 814. More details are in the blog post

Version 3.99 dev series

This was a development release series, leading towards version 4.0. Changes are summarised under that release.

For the full list of issues and changes leading to the 4.0 release, see https://github.com/shawnlaffan/biodiverse/milestone/17

Version 3.1

For the full list of issues and changes leading to the 3.0 release, see https://github.com/shawnlaffan/biodiverse/milestone/16

  • Indices
    • General speedups have been implemented for many of the calculations in the Phylogenetic Indices set.
  • GUI
    • Bugs in the list sorting in the View Labels tab have been fixed. Issue 747
    • Binary version for Macs: the select and pan buttons should now display properly (and seem not to have displayed properly since v2.00). Please report if this is not the case.
  • Trees
    • Exports are now faster for very large trees
    • Plotting trees in the GUI is now faster (although work is still needed - see Issue 754). See Issue 753 as one example.

Version 3.00

For the full list of issues and changes leading to the 3.0 release, see https://github.com/shawnlaffan/biodiverse/issues?utf8=%E2%9C%93&q=milestone%3ARelease_3

  • General
    • Unicode file names are now supported. This was implemented across several issues. (A few) more details are in the blog post
    • The minimum perl version is now 5.22, to take advantage of some of the optimisation and general improvements in the language. This is invisible to users of the exe versions, but users of the source code version need to ensure they use a perl version of 5.22 or later. Issues #680, #705, #659
    • Exports to XML are no longer supported. YAML and JSON do all that is needed here. Issue #736
    • Cluster outputs can be exported to shapefile format. This gives polygons for each branch in the tree. Issue #161. More details are in the blog post
  • BaseData
  • Randomisations
    • These are now faster for large basedata sets. A consequence is that the randomisations for a given PRNG seed value will differ from version 2.1 and earlier, so if exact replication is needed then ensure you use the same version as the analyses you are replicating. Issue #703
  • Cluster and RegionGrower analyses
    • The internal index used for matrices now uses the C locale for numeric values. Incorrect values could otherwise be returned in some locales where the comma is used as the radix character. Biodiverse now throws an exception when it encounters indexes with commas in the values, recommending that the matrix be rebuilt. Issue #742
  • Trees
    • Tree exports to shapefile format are no longer supported. The original purpose is better served exporting to newick with tree branch colours. Issue #735
  • GUI
    • The index lists in open analysis tabs are now updated when a randomisation completes. Previously the tab had to be closed and re-opened. Issue #693
    • Display statistics are updated when an analysis is re-run. Previously the tab had to be closed and re-opened to ensure the correct ranges of values were used. Issue #714
    • Spatial analyses can be exported to RGB GeoTIFF files. This allows users to reconstruct in a GIS package the colour scheme used in Biodiverse. More details in the blog post. Issue #375.

Version 2.99 dev series

This was a development release series, leading towards version 3.0. The summary of changes is under the Version 3.00 entry

For the full list of issues and changes leading to the 3.0 release, see https://github.com/shawnlaffan/biodiverse/issues?utf8=%E2%9C%93&q=milestone%3ARelease_3

Version 2.1

This version provides a small number of updates and improvements over the version 2.0 release.

Highlights are:

  • GUI
    • The label list in the view labels tab is now correctly updated when multiple labels are deleted. Issue #700
    • The user defined colours in the cluster tab uses a 13 colour palette by default (it was 9). Issue #688
  • Exports
  • Randomisations
    • The structured randomisations are faster for larger data sets. Issue #685
  • Tree trimming
    • Tree trimming has been sped up for large trees. Issue #679
    • The trim trees tool has the option to trim to the last common ancestor, thereby removing a dangling root node. Issue #670

For the full list of issues and changes in the 2.1 release, see https://github.com/shawnlaffan/biodiverse/issues?utf8=%E2%9C%93&q=milestone%3ARelease_2.1+

To see the full list of open issues or to report a bug or enhancement request, see https://github.com/shawnlaffan/biodiverse/issues

Version 2.0

This is the 2.0 release. It contains all of the changes in the version 1.99 development series.

This release contains several major changes to the underlying code-base, as well as the addition of several major new features. They are not guaranteed to be backwards compatible with previous versions (1.1 and earlier, see this blog post for more details). It can still use most Biodiverse files created using earlier versions, but files created using this version are not guaranteed to work with earlier versions.

  • The only change of note from the 1.99 series is that a serious memory leak in the Mac implementation has been fixed. Details are in Issue 671.

For the full list of issues and changes leading to the 2.0 release, see https://github.com/shawnlaffan/biodiverse/issues?utf8=%E2%9C%93&q=milestone%3ARelease_2.0+

To see the full list of open issues or to report a bug or enhancement request, see https://github.com/shawnlaffan/biodiverse/issues

Version 1.99 dev series

This is a development release series, leading towards version 2.0.

Collectively, these releases comprise several major changes to the underlying code-base, as well as the addition of a number of major new features. They are not guaranteed to be backwards compatible with previous versions (1.1 and earlier). They can still use most Biodiverse files created using earlier versions, but files created using this version are not guaranteed to work with earlier versions.

For the full list of issues and changes leading to the 2.0 release, see https://github.com/shawnlaffan/biodiverse/issues?utf8=%E2%9C%93&q=milestone%3ARelease_2.0+

To see the full list of open issues or to report a bug or enhancement request, see https://github.com/shawnlaffan/biodiverse/issues

Version 1.99_008

Version 1.99_007

  • Big ticket items
    • Biodiverse now installs cleanly on Macintosh computers. Look for the development release download links here and installation instructions here
    • Biodiverse now provides a remap guesser to make it easier to match labels (e.g. taxa names) between basedata, trees, matrices and property tables. See the blog post for more details.
  • GUI
    • Tree exports can now include the last-used branch colours. Issue 630
  • BaseData
    • Spreadsheet and shapefile imports now properly support DMS (degrees-minutes-seconds) group coordinates. Issue 627
  • Analyses
    • Randomisations now provide rank relative significance scores. Issue 607. More details in the blog post
  • Indices
    • The inter-event interval (IEI) indices have been removed from the main distribution. If needed then they can be made available again. Issue 661
  • Exports
    • File exports now support definition queries. This allows users to export only a subset of their data. Issue 599

Version 1.99_006

Version 1.99_005

  • GUI
  • File formats
    • Some basedatas were not saving correctly when using the Sereal format. This only affected users of the 1.99_004 release and was fixed in commit a68eb20 (which also made it consistent with the existing Storable approach).

Version 1.99_004

  • File formats
    • The default file format now uses Sereal instead of Storable. Issue 358. See this blog post for more details.
  • Randomisations
    • Randomisation outputs can now be renamed. Issue 609
    • Randomisation scores are now automatically converted into rank-relative positions. These can be more easily converted to significance scores. Issue 607. See this blog post for more details.
  • Exports
    • JSON is now supported for table exports. Issue 593

Version 1.99_003

  • GUI
    • The overlays system now warns you if the shapefile is unlikely to be visible, for example when the shapefile is in geographic coordinates but the BaseData file is in an Albers coordinate system. Issue 604
    • The randomisation tab has been restructured to be more compact.
  • Randomisations
  • Indices
    • A "central" variant of the RPE indices is now available and can be used with the PE Central indices. A description is here. The calculation is the same as the non-central version, but the set of branches used are taken from the first neighbour set.
  • Trees

Version 1.99_002

  • GUI
    • No more do we depend on Gtk2::GladeXML for the user interface, thus removing a dependency that has long been deprecated. Issue 413
  • Indices

Version 1.1

This is a small-ish update to the 1.0 release, containing a few bug fixes and enhancements.

The main highlights are:

  • GUI
    • Groups can be renamed in the same way that labels are. This is most useful when you have text based group names, not coordinate based names. Issue 553
    • Two basedatas can be merged, providing they have the same cell sizes and origins. Issue 493
    • View labels tab: Selected labels can be copied to the clipboard. Issue 557 Read the blog post
    • Trees can be ladderised so they plot child branches in order of the number of descendants. Issue 530
  • Data import
  • Analyses
    • Randomisations (read the blog post)
      • Randomisations can now be run for subsets of your data. Specifying a definition query determines which groups are randomised, while specifying a spatial condition constrains the randomisation to stay within subsets. In this way one can randomise labels such that they stay within, for example, the bioregion in which they are found. Issue 554
      • Randomisations now allow users to specify a subset of labels which will be held constant, i.e. their distributions are not randomised. This allows one to, for example, hold one clade in a tree constant, while randomising the remainder of the data. Issue 556
    • The spatial index is now used more effectively. Issue 550, Issue 545, Issue 551
    • Calculations and Indices
      • New calculation for phylogenetic abundance. Issue 559

For the full list of issues and changes leading to the 1.1 release, see https://github.com/shawnlaffan/biodiverse/issues?utf8=%E2%9C%93&q=milestone%3ARelease_1.1+ and https://github.com/shawnlaffan/biodiverse/issues?utf8=%E2%9C%93&q=milestone%3A%22Metadata+system+-+use+OO+not+bare+hashes%22+

To see the full list of open issues or to report a bug or enhancement request, see https://github.com/shawnlaffan/biodiverse/issues

Version 1

This is the 1.0 release. It contains all of the changes in the version 0.99 development series.

This release contains several major changes to the underlying code-base, as well as the addition of several major new features. It is not backwards compatible with previous versions (0.19 and earlier) in several respects. It can still use most Biodiverse files created using earlier versions, but files created using this version are not guaranteed to work with earlier versions.

For the full list of issues and changes leading to the 1.0 release, see https://github.com/shawnlaffan/biodiverse/issues?utf8=%E2%9C%93&q=milestone%3ARelease_1.0+

To see the full list of open issues or to report a bug or enhancement request, see https://github.com/shawnlaffan/biodiverse/issues

Version 0.99 dev series

This is a development release series, leading towards version 1.0.

Collectively, these releases comprise several major changes to the underlying code-base, as well as the addition of a number of major new features. They are not backwards compatible with previous versions (0.19 and earlier) in several respects. They can still use most Biodiverse files created using earlier versions, but files created using this version are not guaranteed to work with earlier versions.

For the full list of issues and changes leading to the 1.0 release, see https://github.com/shawnlaffan/biodiverse/milestones/Release1.0

To see the full list of open issues or to report a bug or enhancement request, see https://github.com/shawnlaffan/biodiverse/issues

Version 0.99_007

This is the seventh phase of the development version leading towards version 1.0.

Main changes since the preceding version (0.99_006) are below. Issue numbers which are not crossed out are yet to be completed.

  • GUI
    • The label selection mode can now be set, so in addition to being able to create a new selection each time one of the grid, tree or matrix panes is clicked, users can now also add to or remove from the current selection. This allows the selection of, for example, distinct clades on the tree. These might then be deleted from the basedata using the deletion methods added in the previous development release. Issue 535
    • Fixed a crash when a tree was trimmed and had no remaining branches, and a View Labels tab was open. Issue 534

Version 0.99_006

This is the sixth phase of the development version leading towards version 1.0.

Main changes since the preceding version (0.99_005) are below. Issue numbers which are not crossed out are yet to be completed.

  • GUI

    • The new pan and zoom functionality now works consistently across all views (maps, trees and matrices). Issue 353
    • New functions have been added to work with the selected labels.
      • Labels can be selected using partial text matches. These use regular expressions, so can be as complex as is needed, but the simplest case is just a fragment of the label name. Selections can optionally be added to or removed from. Issue 529
      • Selected labels can be deleted from the basedata, or new basedatas can be created using the selected (or non-selected) labels. Issue 528.
      • The selected set can be switched (inverted) so all non-selected labels become the selected set. Issue 532
      • The selected set (labels and the groups in which they occur) can be exported directly from the View Labels tab. Issue 414
  • Export/import

    • Trees exported from Biodiverse now roundtrip properly when labels are quoted. Issue 270
    • Sparse format matrix files can now be imported in the GUI. Issue 82
    • Matrix exports now use a progress dialog to avoid a non-responsive GUI. They also write direct to file to reduce memory overheads. Issue 517
    • Basedata imports are now faster. The effect is greatest for raster imports where there is no transform in place. Issue 527
  • Analyses

    • In the spatial analyses, users can control the use of the spatial index across the whole analysis, or on a per-spatial condition level. Issue 205
    • Control is also available for result and neighbour set recycling (the system detects these correctly in most cases, but it is useful to control it in falsely detected cases). Issue 205

Version 0.99_005

This is the fifth phase of the development version leading towards version 1.0.

Main changes since the preceding version (0.99_004) are below. Issue numbers which are not crossed out are yet to be completed.

  • GUI
    • The colour of cells with undefined (nodata) values can now be set by the user. So can the colour of cells which failed the definition query or were otherwise excluded. Issue 278
    • An export menu is now visible in all output tabs so one does not need to go back to the outputs tab whenever one wishes to export them. Issue 273
    • Progress bars are now displayed in all matrix exports. This avoids periods of GUI non-responsiveness. Matrices are also written directly file to reduce memory overheads for large matrices. Issue 517

Version 0.99_004

This is the fourth phase of the development version leading towards version 1.0 (0.99_003 was only used for internal numbering).

Main changes since the preceding version (0.99_002) are below. Issue numbers which are not crossed out are yet to be completed.

  • GUI

    • The phylogenetic endemism and related indices are now in their own category (Phylogentic Endemism). The Phylogenetic Indices category was getting too busy. Issue 499
    • Cell outlines can now be turned off. This is useful when cells are small and any outlines obscure the cell contents. Issue 311
    • The legend can be hidden so it no longer overlaps with the grid. Issue 59
    • Display cursors now change to match the selected mode (e.g. zoom in, pan, select). Issue 490
    • A warning is now shown at startup when extensions cannot be loaded. This was previously only sent to the console window. Issue 500
    • The width of tree branches can now be controlled. The default value of zero will let the system choose a value based on the sparseness of the terminal branches. Issue 505
  • Analyses

    • Analyses are now run as temporary objects and then copied across on success. This means that many of the optimisations where neighbours and matrices are recycled can apply more often since the originals are not replaced until the analysis completes successfully. Issue 444
  • Indices

    • New calculations are now available to calculate the label sample count percentiles across a sample, as well as the rank relative abundance of the labels in the processing group relative to all other groups in the neighbour sets. Issue 507
  • Imports

    • Basedata imports now ignore records with a value of NA. This makes it easier to work with data exported from R as no special processing is needed. Issue 489
    • Basedata imports now have an option to control the number of decimal places used in the group axis calculation. The default is currently 7. Issue 488
  • Exports

    • Tree exports to nexus format can optionally not use the translate block. This means internal nodes can be named and the read.nexus function in ape will still be able to read the file. Issue 502

Version 0.99_002

This is the second phase of the development version leading towards version 1.0.

Main changes since the preceding version (0.99_001) are below. Issue numbers which are not crossed out are yet to be completed.

  • GUI

    • The pan and zoom interface has been rewritten to be more like other tools and to present a cleaner interface. Issue 353
    • Tree plots now grey-out non-highlighted branches. This makes it much easier to see which branches are selected. Issue 464
    • The Spatial and Matrix tabs now have a tree panel which plots the tree used in the analysis, or the tree selected at the project level. This works similarly to the View Labels tab in that branches are highlighted as cells are hovered over, and cells are highlighted as branches are hovered over. Issue 409
    • A popup message is now shown when a basedata has more than two axes, as this could cause overplotting of groups. Issue 461
    • Trees can now be converted to their equal branch length form. Issue 504 (see also Issue 482)
  • Analyses

    • The rand_structured randomisation is now considerably faster. Issue 487
  • Indices and Calculations

    • The relative phylogenetic indices used in the CANAPE process have now been added (see http://dx.doi.org/10.1038/ncomms5473 ). Issue 482
    • Users can now calculate the phylogenetic endemism analogous to the Endemism Central calculation, such that the set of branches considered are those in neighbour set 1 but the local ranges come from across neighbour sets 1 and 2. Issue 460
  • Trees

    • Tree imports are now considerably faster. For example, the Open Tree of Life tree with ~2.5 million nodes takes 3 minutes (note that it needs 6.5GB RAM and the GUI is unlikely to be able to plot it). Issue 483
    • Tree trimming is now considerably faster. Issue 470

Version 0.99_001

This is the first phase of the development version leading towards version 1.0.

Main changes since the previous version (0.19) are below. Issue numbers which are not crossed out are yet to be completed.

  • Analyses

    • The RegionGrower analyses now stop once the maximum optimisation criterion is reached and delete singleton nodes by default. This leaves a smaller tree consisting only of those elements required to optimise the index. Issue 451, Issue 448
    • The Cluster and RegionGrower tie breakers are now optional and are faster and less memory intensive. If they are turned off then the pre-0.19 tie-breaker approach is used. Issue 427
  • Indices and calculations

    • The NRI and NTI indices from PhyloCom are now supported. Note that these reverse the sign compared to PhyloCom, so positive values are dispersed and negative are clustered. This is consistent with the picante package in R. Issue 442
    • New indices are now available for per-node contributions to the PD and PE indices, including their ancestral components. Issue 434
    • Kulczynski 2 is now supported. Issue 445
    • A range weighted Gi* index of label properties is now available. Issue 402
    • The count of terminal nodes used in the neighbour sets can now be obtained. Issue 265
    • The endemism and rarity calculations now return an undefined value when the relevant neighbour set contains no labels (consists only of empty groups). Previously it returns zero. Issue 458
  • Spatial Conditions

    • sp_box() function is now available. Issue 159
  • GUI

    • Exclusions dialogue. Empty groups and labels can optionally not be deleted. This allows the geographic extent to be consistent before and after exclusions are run. Issue 423
    • The progress dialogues are now unified into a single window, greatly reducing the number of popup windows generated in several analyses. Issue 295
    • The label range and sample counts can now be added as label properties. This is useful when one wants to subset a data set using the Exclusions dialogue, but retain their original ranges and abundances. This is accessed via the basedata menu. Issue 412
  • Imports

    • Basedata
      • Raster data files can now be imported directly. Any file format supported by GDAL can be used. Issue 408
      • Shapefiles can now be imported (point formats only). Issue 408
      • When multiple files are selected they can now optionally be assigned to separate Basedatas. Issue 378
    • Trees
      • The tabular tree formats can now be imported. Issue 322
    • Matrices
      • Sparse matrices can be imported (but not yet in the GUI). Issue 82
  • Exports

    • BaseStructs (groups, labels and spatial outputs).
      • Shapefile format (polygons and points) is now supported. Issue 419
      • GeoTIFF is now supported. Issue 447
      • The ER-Mapper exports are no longer offset by half a cell. Issue 453
    • Trees
      • Export to shapefile format is now supported. Issue 410

Version 0.19

This was released on 28-Oct-2013. It is a stable release and contains all changes from the 0.18 development series. See the list here.

To see the full list of issues and changes in this release, see https://github.com/shawnlaffan/biodiverse/issues?utf8=%E2%9C%93&q=label%3AMilestone-Release0.18+

To see the list of open issues or to report a bug or enhancement request, see https://github.com/shawnlaffan/biodiverse/issues

Version 0.18

This was a development version, with periodic releases of test versions.

The last development version was 0.18_008

Main changes since previous version (0.17) are:

  • Analyses
    • Cluster and Region Grower analyses now have user-controllable tie-breaker options. Previously the system would choose a pair at random when there was a choice of two or more pairs. Now users can opt to maximise or minimise any of the indices available for cluster or region grower analyses, in addition to a random selection. For example, one might wish to choose the pair that maximises the weighted endemism score, and if that still results in tie then choose a pair at random. Note that this change also involved changes to the internals of the clustering algorithm, so analyses with the new method will not result in the same set of clusters as before, even if the same PRNG seed is set. However, randomisations of cluster analyses built using the old system will still follow the old approach so they will be valid. Issue #116
    • Cluster analyses: Users can now control the pseudo-random number generator (PRNG) sequence by specifying the PRNG seed. This means that, when a random tie-breaker is used, one can guarantee the same order each time the analysis is run (but see the FAQ). Issue #356
    • Cluster analyses: Analyses using more than one spatial condition now work properly. Previously the clusters would bleed across boundaries they should not. Issue #397
    • The big memory leak in the randomisations has finally been tracked down and fixed. Issue #5
    • Randomisations: New options to perturb the trees and group properties. Issue #388, Issue #389
  • Indices and Calculations
    • Add new calculation to obtain the list of labels that occur on the tree. Issue #319
    • The phylogenetic diversity and phylogenetic endemism measures now use only the labels on the tree. The main change for users is that the PD_per_taxon index will now divide PD by the number of labels in the sample that are on the tree, not the species richness of the sample. Previously, if your sample contained labels not on the tree then the values returned were too low. Issue #320.
    • calc_pd_node_list returns a hash with the node lengths (index PD_INCLUDED_NODE_LIST). Previously it just had values of 1. Issue #321.
    • The PE_WE index is now undefined when no tree branches occur in the neighbour sets.
    • New calculations have been added to replicate some of the analyses in PhyloCom. These are available under the PhyloCom category in the calculations lists. Issue #331 and Issue #332.
    • New calculation to obtain a list of the nodes not on the selected tree. Issue #334
    • The AED and related indices are now calculated correctly. Issue #206
    • New indices for corrected weighted phylogenetic endemism and rarity (PE_CWE and PHYLO_RARITY_CWR). These are phylogenetic analogues of the corrected weighted endemism index and can be interpreted as the degree to which the ranges or abundances of branches found in a neighbour set are restricted to that neighbour set. A value of 1 is completely restricted, values approaching zero have very little restriction.
    • The taxonomic and matrix overlap calculations have been removed. They never worked properly in any case. Issue #400
  • Spatial conditions
    • The sp_match_text() and sp_match_regex() conditions now match against the whole label by default. Specifying the axis argument will make them behave as in previous releases. Issue #325
    • New condition sp_select_element() which will match only one element (group). This is the same as sp_match_text() but has been optimised to make it considerably faster since it can only ever match one element. Issue #326
    • Analyses using point in polygon conditions for neighbour sets 1 & 2 now work correctly. Previously no groups were identified in neighbour set 2. Issue #380
    • The assessment of spatial conditions is now considerably faster. This will substantially speed up analyses with complex conditions. (But note that analyses using sp_self_only() will not show any difference because the system knows it does not need to run any comparisons, thus avoiding needless computation). Issue #381
  • GUI
    • Trees can be rescaled so the branch lengths are divided by their ranges on the selected BaseData. This will help when interpreting the various phylogenetic endemism indices. Issue #385
    • Label and group properties can now be attached after the data are imported. Issue #327
    • Labels can be renamed after import. Issue #349
    • The run exclusions dialogue now supports additional exclusions criteria. These include:
      • Labels can be deleted using a list from a file (issue #348) or using a text match (issue #347).
      • Groups can be excluded using a definition query, using the same syntax as for a spatial analysis (issue #370).
  • Exports
    • Delimited text exports for BaseStruct objects (groups, labels, spatial analyses) now write directly to file, avoiding large memory usage. Issue #350
    • All tree nodes can now be optionally exported when using the Table Grouped type. Issue #312
  • Data structures
    • The matrices are now less memory hungry. Previously matrices with many unique values would require large amounts of storage due to the value indexing used. This was a particular problem for phylogenetic turnover measures. Issue #328

To see the full list of issues and changes in the 0.18_00x versions, see https://github.com/shawnlaffan/biodiverse/issues?utf8=%E2%9C%93&q=label%3AMilestone-Release0.18+

To see the list of open issues or to report a bug or enhancement request, see https://github.com/shawnlaffan/biodiverse/issues

Version 0.17

This version was released on 06Jul2012.

Main changes since the previous version (0.16) are:

  • GUI
    • Shapefile overlays are selected as soon as they are opened. This ensures they are plotted by default. Issue #293
    • View labels: numeric label data sets are sorted numerically. Issue #260
  • Analyses
    • Calculations for cluster nodes can now be done after the event. The system no longer needs to completely rebuild the tree. Issue #289
  • Calculations and indices
  • Spatial conditions
    • Added sp_point_in_poly_shape() to use polygons from a shapefile. Issue #226
    • Added optimisation for conditions that always return the same result. Issue #304
    • Added sp_get_spatial_output_list_value() to allow access to other outputs in the same BaseData. This allows one to, for example, restrict analyses to groups with an endemism score above some threshold. Issue #233
  • Exports
    • Users can now add plot coords to the tabular tree exports. This allows reconstruction of the tree in, for example, a GIS. Issue 281
    • NA is now an option for nodata. This improves compatibility with R. Issue #271
    • Raster exports now work for all cases (was getting errors for some cell sizes). Issue #294
  • Imports
    • Tree imports now work when there are duplicate node names. Any duplicates have __dup$i appended to the name, where $i is an integer that is incremented from 1. For a node called node, any duplicates are called node__dup1, node__dup2, etc. Issue #302

To see the full list of issues and changes in this version, see https://github.com/shawnlaffan/biodiverse/issues?utf8=%E2%9C%93&q=label%3AMilestone-Release0.17+

To see the list of open issues or to report a bug or enhancement request, see https://github.com/shawnlaffan/biodiverse/issues

Version 0.16

This version was a series of beta releases.

Main changes since the previous version (0.15) are:

  • General
    • Groups and spatial outputs can now be exported direct to DIVAGIS raster formats. Issue #220.
  • GUI
    • Matrices generated by cluster analyses are now added to the project and can be visualised as a spatial plot. Click on an element (cell) in the plot to see its dissimilarity with every other element used in the matrix (the index element is coloured grey). This is very useful when used in conjunction with tools like Generalised Dissimilarity Modelling and when interpreting correlograms of species turnover. Issue #199.
    • Grey scale shading is now supported. Issue #32.
    • Users can now switch between tabs using control-tab and shift-control-tab keys. Issue #196.
    • The colour of the shapefile overlay can be changed. Issue #75.
    • Outputs tab - the output type is listed next to each output. Issue #201.
    • Colour stretches can be adjusted using percentile values (2.5, 5, 95, 97.5). Issue #244.
    • View labels tab
      • Element properties with a value of nodata are displayed as -99999. Issue #189.
      • System no longer intermittently hangs when control clicking. Issue #194.
      • Labels list is now stable when sorting by a column with tied values. Issue #246.
      • Label and group axes can be re-ordered after import. This is useful when you have forgotten to reorder them at import and, for example, your data file contains a field for latitude before longitude. Issue #188.
  • Analyses
    • New analysis type of RegionGrower. It is an extension to the cluster analyses that uses any scalar metric that lumps two neighbour sets together when calculated. One can also merge those pairs that either maximise or minimise the selected index. When used with indices like richness it is effectively a one-pass complementarity analysis. (Its matrices can also be displayed spatially since it is just a variant on the cluster analyses - see issue #199). Issue #204.
    • Cluster matrices can be written to file as they are built. These are not added to the GUI, thus saving memory and allowing extremely large matrices to be built for use in external applications. Issue #186.
    • More efficient cluster matrix construction and use. Issue #185, Issue #210, Issue #234
    • Better handling of empty groups. Issue #180, Issue #181, Issue #228, Issue #234.
  • Calculations and indices
    • NEST_RESULTANT index is now correctly calculated. Issue #182.
    • Added phylogenetic nearest taxon distance indices. Issue #225.
    • Indices for phylogenetic dissimilarity are now available. Issue #215. Issue #240.
    • Added AED, BED and related indices. Issue #206.
    • Added indices for numeric label dissimilarity. Issue #223.
  • Spatial conditions.
    • sp_select_sequence() now works as a definition query. Issue #190.
    • sp_is_left_of(), sp_is_right_of() and sp_in_line_with() identify if the element is to one side of a vector or on it. Issue #202.
    • sp_select_block () for block subsampling. Issue #218.
    • sp_point_in_polygon(). Issue #221.
    • sp_group_not_empty() - does the group have any labels? See Issue #234.

To see the full list of issues and changes, see https://github.com/shawnlaffan/biodiverse/issues?utf8=%E2%9C%93&q=label%3AMilestone-Release0.16+

Version 0.15

Main changes since the previous version (0.14) are:

  • Calculations and Indices
    • The PD node list has been moved to its own calculation (issue #130)
    • The Phylogenetic Endemism (PE) lists have been moved to their own sub (issue #131)
  • Exporting data
    • Exporting groups to raster formats now works properly (issue #129)
    • Exporting matrices from a cluster analyses now works (issue #142)
    • Exporting to ER-Mapper files now works properly, with some caveats (see issue #135 and the FAQ)
  • Importing data
    • Column numbers are now displayed when importing data (issue #128)
    • Tree imports now use the same quotes character as BaseData imports (issue #152)
  • Spatial Conditions
  • Randomisations
  • Visualisation
    • User defined properties are now displayed in the view labels tab (issue #155).
  • Under the bonnet
    • User defined properties are now imported properly in the GUI (issue #154)
    • Neighbour set recycling works properly now (issue #127 & issue #145)
    • Module Statistics::Descriptive2 is now deprecated. It has been replaced by Biodiverse::Statistics which depends explicitly on Statistics::Descriptive (issue #139)

To see the full list of changes, see https://github.com/shawnlaffan/biodiverse/issues?utf8=%E2%9C%93&q=label%3AMilestone-Release0.15+

Version 0.14

Main changes since the previous version (0.13) are:

  • GUI (and general)
    • The naming errors with saving basedata, tree, and matrix files to the biodiverse native format have been corrected. (Issue #104)
    • Reading a tree file from biodiverse format works again. (Issue #105)
    • The GUI now recognises R style tables for import (as exported using the write.table() function in R). (Issue #20)
    • Tree and matrix objects can be renamed. (Issue #72)
    • Users can now describe the selected basedata, tree and matrix. Results are printed to both a popup and to the log window (from where they can be more easily copied). (Issue #93)
    • The parameters sections of the Spatial and Cluster tabs can be hidden to free up real estate when selecting calculations to run. Click on the Parameters button at the top left of the tab to apply it. (Issue #68)
    • Trees and matrices embedded in basedata objects can be added to the project. (Issue #71)
    • Labels in a basedata object can be deleted using the nodes/elements in the selected tree or matrix. The converse can also be done, where those BaseData labels not in the matrix or tree are deleted. (issue #74).
  • Calculations and indices:
  • Spatial Conditions
    • New condition sp_match_regex which allows the user to match using arbitrary regular expressions. For example, to match any group where the neighbour's third axis starts with the processing group's third axis, use sp_match_regex (re => qr/^$coord[2]/, axis => 2, type => 'nbr'). (Issue #101)
  • Randomisations
    • The randomisations now compare against all lists, not just those in SPATIAL_RESULTS. One consequence of this is that the resulting list names have changed. To enable users to keep track of the results across multiple lists, the results are now named using the randomisation name, followed by >>, followed by the original list name. For example, for a randomisation called Rand1, one could have resulting lists called Rand1>>SPATIAL_RESULTS and Rand1>>ENDC_WTLIST. The naming scheme for the list contents has not changed (see AnalysisTypes#Randomisations). The code is not backwards compatible, so previously run randomisations cannot be extended cleanly using this method. This is because the SPATIAL_RESULTS comparisons will be divided across two lists in an output, e.g. Rand1 and Rand1>>SPATIAL_RESULTS. Re-running the randomisation from scratch is recommended. Alternately, one can manually sum the C_ and Q_ values across the two result sets and from them calculate the updated P_ values. (Issue 100)
    • The swapping algorithm has been rewritten so it converges more efficiently for large data sets. (Swapping is used to reach richness targets in the rand_structured randomisation). A consequence of this is that structured randomisations will not produce the same result between versions 0.13 and 0.14 if the seed value is specified. This is because the random values are used in a different order and so will produce different results if given the same sequence of random values. As with the previous change, the simple solution to this is to start any randomisations afresh after upgrading rather than continuing any existing randomisations. Continuing randomisations created in BaseData objects prior to version 0.14 will result in mixed algorithms, thus making reproduction of results difficult. (Issue #103)
    • The comparison algorithm used when comparing spatial analyses now uses recycled results, resulting in a smaller memory footprint. Recycling occurs when results are the same across the neighbourhood, as occurs with block and zone type spatial conditions. In these cases one can generate one set of results and apply them across the neighbourhood rather than having multiple sets of identical results. (Issue #107)
  • Under the bonnet
    • Several optimisations have been added where the system can use recycled results (those where the results are the same for all groups in a neighbourhood).

To see the full list of changes, see https://github.com/shawnlaffan/biodiverse/issues?utf8=%E2%9C%93&q=label%3AMilestone-Release0.14+

Version 0.13

01Feb2010.

Main changes since the previous version (0.12) are:

  • Analyses, Calculations and Indices.

    • The use of the term "analyses" in the documentation is now clearer. Analyses can be one of Cluster, Spatial or Randomisation. The term "calculation" is now used to describe the collection of indices that are calculated together, e.g. for the groups under a cluster node or within a spatial neighbourhood. An analysis can apply several calculations to generate some number of indices.
    • The endemism calculations have been subdivided. The list indices are now in their own calculations, and need to be called explicitly if needed.
    • Two new rarity calculations have been added to generate the lists of weights and sample counts for each label used, as per the endemism calculations.
  • View labels tab:

    • The view labels lists now have additional columns to allow sorting based on the selections (i.e. to promote the selected set to the top of the list). Note that the list order changes dynamically according to the sort options. This means that the list will be reordered if the label selection is changed and there is a sort in place using one or both of these columns.
    • The second view labels list is only shown if there is a selected matrix. This is because it is used to show the matrix column selection and control their sort order.
    • The map legend is now shown in the view labels map pane. This allows easier interpretation of the mapped values.
  • Trees

    • Nexus file import is now more flexible in the characters it accepts in names (anything that has no special meaning in nexus files), and also numeric formats used for lengths.
    • Newick files are now supported for import (see issue #79).
    • When displaying the cluster dendrogram, the user can choose not to use the slider bar to select nodes to plot on the map (access via the dendrogram Options menu).
  • Group coordinates can be specified as Degree, Minutes, Seconds within a single column. (Note: this is currently a beta level implementation). See issue #61.

  • The system now warns if the user attempts to create a new output, or rename an existing output, using a name that is already in use for that output type (see issue #62).

  • The randomisation list results are now accessible (see issue #45).

  • See also https://github.com/shawnlaffan/biodiverse/issues?utf8=%E2%9C%93&q=label%3AMilestone-Release0.13+

  • For the list of current issues, or to submit a bug report or enhancement request then please see https://github.com/shawnlaffan/biodiverse/issues

Version 0.12

07Dec2009

Main changes since the previous version (0.11) are:

  • The Windows version comes bundled with updated GTK libraries, and the executable version does not require any additional downloads. (We are still working on an easy Mac install).

  • The export interface has been revamped to use two windows. The first allows the selection of the output format and the second has parameters specific to that format. This makes the parameter choices clearer as irrelevant parameters are not displayed.

  • Feedback to GUI does not contain the line numbers for the code. They are still printed to the log window, though.

  • Allow native format BaseData, matrix and tree files to be loaded on startup (previous versions only allowed a project file to be loaded as an argument on startup).

  • User is warned if they try to add a new spatial or cluster output to a BaseData and it has one or more randomisations.

  • Randomisation outputs can now be deleted.

  • Randomisation outputs now have export methods to generate text files of their initial and current pseudo-random number generator (PRNG) state.

  • The MXO_WARD and TXO_WARD metrics have been removed. Use the new "Compare dissimilarity matrix values" analysis instead.

  • Matrix elements (labels) can be remapped on import, as well as excluded/included using the properties table.

  • The scree plot below the dendrogram is minimised to begin with. It can be pulled up when needed.

  • See also https://github.com/shawnlaffan/biodiverse/issues?utf8=%E2%9C%93&q=label%3AMilestone-Release0.12+

Version 0.11

06Nov2009

Main changes since previous version (0.10). This is a cleanup and bug fix release with no new features.

  • Improved feedback and error trapping when running analyses. If the system fails then a popup window will tell you what went wrong and where. This information is still printed to the log window so you can access it after closing the popup.
  • The error trapping also corrects issues where users are unable to close a tab if an analysis fails.
  • Basedata imports fail with a warning when they reach a record with an undefined or non-numeric value to be used for the groups (unless it is a text group). The user can set an option to skip these records (this is on by default).
  • System warns user when trying to export a randomisation output. These cannot currently be exported (although the PRNG state would be useful). The results can be exported from the relevant spatial or cluster output. See KeyConcepts#Randomisations
  • Exports for spatial outputs, groups and labels now have an option to specify the type of file instead of relying on the file extension for this. This does not change the possibilities, but does make them clearer.

Version 0.10

21Oct2009

Main changes since previous version (0.9.1185)

Version 0.9.1185

Main changes since previous release (version 0.9.1127)

  • Installation instructions updated, particularly for the Mac installs.
  • Fixed bug in tree reading module. Now correctly reads nexus format files, including those exported from Biodiverse.
  • Documentation updated.
  • Example files added to distribution (data folder)
  • BaseData import now handles text with embedded newlines (these occur occasionally in plant and other data bases, usually in the collection descriptions).
  • Statistics::Descriptive2 now included in the distribution (under the lib folder)
  • Fixed issue where saved project file is given the name "1.bps"
Clone this wiki locally