This is a tutorial/explanation of how to set up XGBoost for imbalanced classification while tuning for imbalanced data.

There are three main sections:

1. Hyperopt/Bayesian Hyperparameter Tuning
2. Focal and Crossentropy losses
3. XGBoost Parameter Meanings

(references are dropped as-needed)

Hyperopt

The hyperopt package is associated with Bergstra et. al.. The authors argued that the performance of a given model depends both on the fundamental quality of the algorithm as well as details of its tuning (also known as its hyper-parameters).

This is going to get very weird, very quickly. When you create a class in Python, it looks about like the following:

class MyClass:
    pass

Now, let’s say I create some really cool class, with a set of cool functions, but I expect my users to implement some of the functions:

from abc import abstractmethod

class BaseClass:
    @abstractmethod
    def foo(self,):
        raise NotImplementedError

So the intention is, when my user inherits the above class, they do the following:

In a previous post, I detailed how to maintain encapsulation using Python’s property. In this piece, I go through how/why to manage and apply validation to class attributes in an object-oriented fashion by means of a fairly plausible example.

A type is the parent class of class, therefore any class is actually a sub-type of type. The following are equivalent:

a = int(8)
a = 8
type(a) # python knows to create an int without being explicit

    int

The point of implementing custom attribute types is (in my case), for validation. The general pattern for creating a class that serves as a type to validate instance attributes is as follows (for a descriptor):

If you ever created a class in Python, you probably accessed it using dot notation (i.e. instance_name.attribute_name).

That’s python’s way of calling getattr by means of an alias:

class A:
    var = 10
    pass

a = A()
# this is how Python accesses attributes
getattr(a, 'var')

10

a.__getattribute__('var') # above is an alias for this

10

The most “pythonic” way of getting and setting attributes is using dot notation:

A.var = 11
print(A.var)

11

which is short for the dunder getattribute method

A decorator is the gateway drug into the world of Python metaprogramming. In python, everything, everything, is an object (specifically a dictionary but let’s not go there). That means that we can pass in and return any object regardless of its types, especially regardless of its type.

If I define a function:

def fn(*args, **kwargs):
    pass

and now call type on fn

type(fn)

function

the type is function (No surprises there). But remember, we can return anything. So if I really wanted to, I could do the following: