... | ... | @@ -2,30 +2,45 @@ |
|
|
|
|
|
An implementation of L2-regularized logistic regression for two-class classification.
|
|
|
|
|
|
```prolog
|
|
|
:- use_module('path/to/.../src/methods/adaboost/adaboost.pl').
|
|
|
|
|
|
%% usage example
|
|
|
TrainData = [5.1,3.5,1.4, 4.9,3.0,1.4, 4.7,3.2,1.3, 4.6,3.1,1.5],
|
|
|
TestData = [3,2,0, 5,1,4, 0,0,4, 3,3,5, 0,5,5, 2,5,5],
|
|
|
adaboost_initModelWithTraining(TrainData, 3, [0,1,0,1], 2, perceptron, 50, 0.0001),
|
|
|
adaboost_classify(TestData, 3, PredictionList, ProbabilitiesList, _).
|
|
|
```
|
|
|
|
|
|
# Available Predicates
|
|
|
|
|
|
* [initModelNoOptimizer/6](/PrologMethods/Classification/logistic_regression#initmodelnooptimizer6)
|
|
|
* [initModelWithOptimizer/7](/PrologMethods/Classification/logistic_regression#initmodelwithoptimizer7)
|
|
|
* [classifyPoint/4](/PrologMethods/Classification/logistic_regression#classifypoint4)
|
|
|
* [classifyLabels/6](/PrologMethods/Classification/logistic_regression#classifylabels6)
|
|
|
* [classifyProbs/6](/PrologMethods/Classification/logistic_regression#classifyprobs6)
|
|
|
* [computeAccuracy/7](/PrologMethods/Classification/logistic_regression#computeaccuracy7)
|
|
|
* [computeError/6](/PrologMethods/Classification/logistic_regression#computeerror6)
|
|
|
* [train/6](/PrologMethods/Classification/logistic_regression#train6)
|
|
|
* [logistic_regression_initModelNoOptimizer/4](/PrologMethods/Classification/logistic_regression#logistic_regression_initmodelnooptimizer4)
|
|
|
* [logistic_regression_initModelWithOptimizer/9](/PrologMethods/Classification/logistic_regression#logistic_regression_initmodelwithoptimizer9)
|
|
|
* [logistic_regression_classifyPoint/3](/PrologMethods/Classification/logistic_regression#logistic_regression_classifypoint3)
|
|
|
* [logistic_regression_classifyMatrix/6](/PrologMethods/Classification/logistic_regression#logistic_regression_classifymatrix6)
|
|
|
* [logistic_regression_computeAccuracy/5](/PrologMethods/Classification/logistic_regression#logistic_regression_computeaccuracy5)
|
|
|
* [logistic_regression_computeError/4](/PrologMethods/Classification/logistic_regression#logistic_regression_computeerror4)
|
|
|
|
|
|
---
|
|
|
|
|
|
[links/resources](/PrologMethods/Classification/logistic_regression#connected-linksresources)
|
|
|
|
|
|
## **_initModelNoOptimizer/6_**
|
|
|
## **_logistic_regression_initModelNoOptimizer/4_**
|
|
|
|
|
|
Initialize the logistic_regression model without specifing a optimizer.
|
|
|
|
|
|
```prolog
|
|
|
%% part of the predicate definition
|
|
|
initModelNoOptimizer( +pointer(float_array), +integer, +integer,
|
|
|
%% predicate definition
|
|
|
logistic_regression_initModelNoOptimizer(DataList, DataRows, ResponsesList, Lambda) :-
|
|
|
Lambda >= 0.0,
|
|
|
convert_list_to_float_array(DataList, DataRows, array(Xsize, Xrownum, X)),
|
|
|
convert_list_to_float_array(ResponsesList, array(Ysize, Y)),
|
|
|
initModelNoOptimizerI(X, Xsize, Xrownum, Y, Ysize, Lambda).
|
|
|
|
|
|
%% foreign c++ predicate definition
|
|
|
foreign(initModelNoOptimizer, c, initModelNoOptimizerI( +pointer(float_array), +integer, +integer,
|
|
|
+pointer(float_array), +integer,
|
|
|
+float32).
|
|
|
+float32)).
|
|
|
```
|
|
|
|
|
|
### Parameters
|
... | ... | @@ -37,16 +52,27 @@ initModelNoOptimizer( +pointer(float_array), +integer, +integer, |
|
|
|
|
|
---
|
|
|
|
|
|
## **_initModelWithOptimizer/7_**
|
|
|
## **_logistic_regression_initModelWithOptimizer/9_**
|
|
|
|
|
|
Initialize the logistic_regression model and specify the optimizer.
|
|
|
|
|
|
```prolog
|
|
|
%% part of the predicate definition
|
|
|
initModelWithOptimizer( +pointer(float_array), +integer, +integer,
|
|
|
%% predicate definition
|
|
|
logistic_regression_initModelWithOptimizer(DataList, DataRows, ResponsesList, Responses, Lambda, MaxIterations, Tolerance, StepSize, BatchSize) :-
|
|
|
Lambda >= 0.0,
|
|
|
MaxIterations >= 0,
|
|
|
Tolerance >= 0.0,
|
|
|
StepSize >= 0.0,
|
|
|
BatchSize > 0,
|
|
|
convert_list_to_float_array(DataList, DataRows, array(Xsize, Xrownum, X)),
|
|
|
convert_list_to_float_array(ResponsesList, array(Ysize, Y)),
|
|
|
initModelWithOptimizerI(X, Xsize, Xrownum, Y, Ysize, Responses, Lambda, MaxIterations, Tolerance, StepSize, BatchSize).
|
|
|
|
|
|
%% foreign c++ predicate definition
|
|
|
foreign(initModelWithOptimizer, c, initModelWithOptimizerI( +pointer(float_array), +integer, +integer,
|
|
|
+pointer(float_array), +integer,
|
|
|
+string,
|
|
|
+float32).
|
|
|
+float32, +integer, +float32, +float32, +integer)).
|
|
|
```
|
|
|
|
|
|
### Parameters
|
... | ... | @@ -59,15 +85,22 @@ initModelWithOptimizer( +pointer(float_array), +integer, +integer, |
|
|
|
|
|
---
|
|
|
|
|
|
## **_classifyPoint/4_**
|
|
|
## **_logistic_regression_classifyPoint/3_**
|
|
|
|
|
|
Classify the given point.
|
|
|
|
|
|
```prolog
|
|
|
%% part of the predicate definition
|
|
|
classifyPoint( +pointer(float_array), +integer,
|
|
|
%% predicate definition
|
|
|
logistic_regression_classifyPoint(PointList, DecisionBoundary, PredicLabel) :-
|
|
|
DecisionBoundary >= 0.0,
|
|
|
DecisionBoundary =< 1.0,
|
|
|
convert_list_to_float_array(PointList, array(Xsize, X)),
|
|
|
classifyPointI(X, Xsize, DecisionBoundary, PredicLabel).
|
|
|
|
|
|
%% foreign c++ predicate definition
|
|
|
foreign(classifyPoint, c, classifyPointI( +pointer(float_array), +integer,
|
|
|
+float32,
|
|
|
[-integer]).
|
|
|
[-integer])).
|
|
|
```
|
|
|
|
|
|
### Parameters
|
... | ... | @@ -79,15 +112,25 @@ classifyPoint( +pointer(float_array), +integer, |
|
|
|
|
|
---
|
|
|
|
|
|
## **_classifyLabels/6_**
|
|
|
## **_logistic_regression_classifyMatrix/6_**
|
|
|
|
|
|
Classify the given points, returning the predicted labels for each point.
|
|
|
|
|
|
```prolog
|
|
|
%% part of the predicate definition
|
|
|
classifyLabels( +pointer(float_array), +integer, +integer,
|
|
|
%% predicate definition
|
|
|
logistic_regression_classifyMatrix(DataList, DataRows, PredictionList, ProbsList, ZCols, DecisionBoundary) :-
|
|
|
DecisionBoundary >= 0.0,
|
|
|
DecisionBoundary =< 1.0,
|
|
|
convert_list_to_float_array(DataList, DataRows, array(Xsize, Xrows, X)),
|
|
|
classifyMatrixI(X, Xsize, Xrows, Y, Ysize, Z, ZCols, ZRows, DecisionBoundary),
|
|
|
convert_float_array_to_list(Y, Ysize, PredictionList),
|
|
|
convert_float_array_to_2d_list(Z, ZCols, ZRows, ProbsList).
|
|
|
|
|
|
%% foreign c++ predicate definition
|
|
|
foreign(classifyMatrix, c, classifyMatrixI( +pointer(float_array), +integer, +integer,
|
|
|
-pointer(float_array), -integer,
|
|
|
+float32).
|
|
|
-pointer(float_array), -integer, -integer,
|
|
|
+float32)).
|
|
|
```
|
|
|
|
|
|
### Parameters
|
... | ... | @@ -99,34 +142,24 @@ classifyLabels( +pointer(float_array), +integer, +integer, |
|
|
|
|
|
---
|
|
|
|
|
|
## **_classifyProbs/6_**
|
|
|
|
|
|
Classify the given points, returning class probabilities for each point.
|
|
|
|
|
|
```prolog
|
|
|
%% part of the predicate definition
|
|
|
classifyProbs( +pointer(float_array), +integer, +integer,
|
|
|
-pointer(float_array), -integer, -integer).
|
|
|
```
|
|
|
|
|
|
### Parameters
|
|
|
| Name | Type | Description | Default |
|
|
|
|------|------|-------------|---------|
|
|
|
| data | +matrix | Set of points to classify. | - |
|
|
|
| probabilities | -matrix | Class probabilities for each point (output). | - |
|
|
|
|
|
|
---
|
|
|
|
|
|
## **_computeAccuracy/7_**
|
|
|
## **_logistic_regression_computeAccuracy/5_**
|
|
|
|
|
|
Compute the accuracy of the model on the given predictors and responses, using the given decision boundary.
|
|
|
|
|
|
```prolog
|
|
|
%% part of the predicate definition
|
|
|
computeAccuracy( +pointer(float_array), +integer, +integer,
|
|
|
%% predicate definition
|
|
|
logistic_regression_computeAccuracy(DataList, DataRows, ResponsesList, DecisionBoundary, Accuracy) :-
|
|
|
DecisionBoundary >= 0.0,
|
|
|
DecisionBoundary =< 1.0,
|
|
|
convert_list_to_float_array(DataList, DataRows, array(Xsize, Xrownum, X)),
|
|
|
convert_list_to_float_array(ResponsesList, array(Ysize, Y)),
|
|
|
computeAccuracyI(X, Xsize, Xrownum, Y, Ysize, DecisionBoundary, Accuracy).
|
|
|
|
|
|
%% foreign c++ predicate definition
|
|
|
foreign(computeAccuracy, c, computeAccuracyI( +pointer(float_array), +integer, +integer,
|
|
|
+pointer(float_array), +integer,
|
|
|
+float32,
|
|
|
[-float32]).
|
|
|
[-float32])).
|
|
|
```
|
|
|
|
|
|
### Parameters
|
... | ... | @@ -139,15 +172,21 @@ computeAccuracy( +pointer(float_array), +integer, +integer, |
|
|
|
|
|
---
|
|
|
|
|
|
## **_computeError/6_**
|
|
|
## **_logistic_regression_computeError/4_**
|
|
|
|
|
|
Compute the error of the model.
|
|
|
|
|
|
```prolog
|
|
|
%% part of the predicate definition
|
|
|
computeError( +pointer(float_array), +integer, +integer,
|
|
|
%% predicate definition
|
|
|
logistic_regression_computeError(DataList, DataRows, ResponsesList, Error) :-
|
|
|
convert_list_to_float_array(DataList, DataRows, array(Xsize, Xrownum, X)),
|
|
|
convert_list_to_float_array(ResponsesList, array(Ysize, Y)),
|
|
|
computeErrorI(X, Xsize, Xrownum, Y, Ysize, Error).
|
|
|
|
|
|
%% foreign c++ predicate definition
|
|
|
foreign(computeError, c, computeErrorI( +pointer(float_array), +integer, +integer,
|
|
|
+pointer(float_array), +integer,
|
|
|
[-float32]).
|
|
|
[-float32])).
|
|
|
```
|
|
|
|
|
|
### Parameters
|
... | ... | @@ -159,32 +198,12 @@ computeError( +pointer(float_array), +integer, +integer, |
|
|
|
|
|
---
|
|
|
|
|
|
## **_train/6_**
|
|
|
|
|
|
Train the logistic_regression model on the given input data.
|
|
|
|
|
|
```prolog
|
|
|
%% part of the predicate definition
|
|
|
train( +pointer(float_array), +integer, +integer,
|
|
|
+pointer(float_array), +integer,
|
|
|
+string).
|
|
|
```
|
|
|
|
|
|
### Parameters
|
|
|
| Name | Type | Description | Default |
|
|
|
|------|------|-------------|---------|
|
|
|
| data | +matrix | Input training variables. | - |
|
|
|
| responses | +vector | Outputs results from input training variables. | - |
|
|
|
| optimizer | +string | The optimizer to use: "lbfgs", "psgd" | lbfgs |
|
|
|
|
|
|
---
|
|
|
|
|
|
# Connected Links/Resources
|
|
|
|
|
|
If you want a more detailed explanation, then go to the python documentation. There is most of the time a good explanation on how the methods work and what the parameters do.
|
|
|
|
|
|
* [**MLpack::logistic_regression_C++\_documentation**](https://www.mlpack.org/doc/stable/doxygen/classmlpack_1_1regression_1_1LogisticRegression.html)
|
|
|
* [**MLpack::logistic_regression_Python_documentation**](https://www.mlpack.org/doc/stable/python_documentation.html#logistic_regression)
|
|
|
* [**MLpack::logistic_regression_C++\_documentation**](https://www.mlpack.org/doc/mlpack-3.4.2/doxygen/classmlpack_1_1regression_1_1LogisticRegression.html)
|
|
|
* [**MLpack::logistic_regression_Python_documentation**](https://www.mlpack.org/doc/mlpack-3.4.2/python_documentation.html#logistic_regression)
|
|
|
|
|
|
added some of the links from the python documentation
|
|
|
|
... | ... | |