-
-
Notifications
You must be signed in to change notification settings - Fork 5.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
outstanding balance function #3059
base: develop
Are you sure you want to change the base?
Conversation
- comments for analyze_trade_parallelism
065dd75
to
dcd85e3
Compare
i still struggle to see why we need this - and what exactly this delivers. I think this is a rather odd approach to calculate this - and being honest, i think the formula used is wrong as in theory, all you should need to do is:
The calculation ( |
The point is to show how much volatility there was intra trade. You only know that each trade was
I did it this way also for speed, this is very fast (although it also uses mostly numpy)
this doesn't work if you have position stacking enabled, you need to keep track of the duration of each trade..unless I am mistaken the
this is what's done at the end
you're right in the sense that calculating it this way, it assumes an amount of |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've been trying to engineer a case where i can see the output or test how this works for at least the last hour, but neither with data from a database (loaded via load_trades_from_db()
), nor with data from a Backtest-result (loaded via load_backtest_data()
) i've been able to get any output other than a one-line dataframe, with 0 balance and the date of the last day of the first month, so i'll have to assume it's somehow broken / non-functioning.
Please provide us with a working example on how this can be used so we can play and test this - otherwise we'll have to close this as it's currently non-functioning - at least in any way i tried to call it).
This can be either via a test-case which asserts the output (this should show us how to call it and how it works) - or via a small case in a comment / notebook so we can test this.
I added an implementation as a loss function based on sortino. Note that hyperopt needs to pass the holcv data... |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Due to the implementation of this (looping for hops_max) this is very slow.
Running this for a timeframe of 5m for 10 pairs for ~1 month takes south of >12s (after fixing the timeframe bug, obviously).
With that kind of performance, using this for hyperopt is near impossible.
I'm also still not 100% certain what this should return - is it the balance (in stake currency) at every moment in time?
i think a calculation should work too - while avoiding the "max_hops" loop and therefore beeing rather performant (<1s for the same dataset than the above test).
def calculate_outstanding_balance_1(
results: pd.DataFrame,
timeframe: str,
min_date: datetime,
max_date: datetime,
hloc: Dict[str, pd.DataFrame],
slippage=0,
) -> pd.DataFrame:
"""
Sums the value of each trade (both open and closed) on each candle
:param results: Results Dataframe
:param timeframe: Frequency used for the backtest
:param min_date: date of the first trade opened (results.open_time.min())
:param max_date: date of the last trade closed (results.close_time.max())
:param hloc: historical DataFrame of each pair tested
:slippage: optional profit value to subtract per trade
:return: DataFrame of outstanding balance at each timeframe
"""
stake_amount = 0.1
# Assume a stake amount of 0.1 (should be a parameter)
results['amount'] = stake_amount / results['open_rate']
from freqtrade.exchange import timeframe_to_minutes
timeframe_min = timeframe_to_minutes(timeframe)
dates = [pd.Series(pd.date_range(row[1].open_time, row[1].close_time,
freq=f"{timeframe_min}min", tz='UTC'))
for row in results[['open_time', 'close_time']].iterrows()]
deltas = [len(x) for x in dates]
dates1 = pd.Series(pd.concat(dates).values, name='date')
df2 = pd.DataFrame(np.repeat(results.values, deltas, axis=0), columns=results.columns)
# df2 is a dataframe containing one row per candle for the duration the trade was open
df3 = pd.concat([dates1, df2], axis=1)
df3 = df3.set_index('date')
# df3 has the dates set correctly (one date per candle / trade). Date is only unique per pair.
values = {}
for pair in hloc:
ohlc = hloc[pair].set_index('date')
df_pair = df3.loc[df3['pair'] == pair]
# filter on pair and convert dateindex to utc
# * Temporary workaround
df_pair.index = pd.to_datetime(df_pair.index, utc=True)
# Combine trades with ohlc data
df4 = df_pair.merge(ohlc, left_on=['date'], right_on=['date'])
# Calculate the value at each candle
df4['current_value'] = df4['amount'] * df4['open']
# 0.002 -> slippage / fees
df4['value'] = df4['current_value'] - df4['current_value'] * 0.002
values[pair] = df4
# display(df4[["pair", 'profit_percent', 'open_time', 'close_time', 'open_rate', 'close_rate', 'open', 'close', 'amount', 'value']].tail())
balance = pd.concat([df[['value']] for k, df in values.items()])
return balance
Results seem similar (the graphical representation is similar) to your results - but i think the scale is different... or some other detail i'm currently missing.
(this is the reason this took a while - as i had to find some uninterrupted time to work on this ...)
freqtrade/data/btanalysis.py
Outdated
timedelta = pd.Timedelta(timeframe) | ||
|
||
date_index: pd.DatetimeIndex = pd.date_range( | ||
start=min_date, end=max_date, freq=timeframe, normalize=True |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is wrong when used for short timeframes.
Assuming "5m" - if that's passed to pd.date_range - it'll create an index with a frequency of "5M" - which is 5 months.
what should be used instead is something along the following lines (a sample of this can be found in analyze_trade_parallelism).
from freqtrade.exchange import timeframe_to_minutes
timeframe_min = timeframe_to_minutes(timeframe)
...
freq=f"{timeframe_min}min"
I imagine that doesn't work with position stacking?
This makes me think so |
it should work with position stacking as well ... it was just a quick comment thrown there to explain what it does ... |
A couple of things I noticed:
Yes I would resample the values like this since you already used abs values balance = (
balance.resample(freq).agg({"value": sum}).asfreq(freq)
) and maybe do a cumsum and plot it like plot-profit
It was the heaviest part, that's also why there was the other loop which was of similar perf. iirc I was testing with |
Cumsum cannot be used on this - the output is the total balance in trades at any point in time ... I'm also not certain on the usefulness of this in a loss function - all it'll do is analyze how much money was in a trades at any point in time - it'll not give you the account balance at that moment, so based on this function, you can't determine if the bot is successful or not. This function can however be useful to analyze how much money is in trades at any point in time - so you might determine to use a lower (or higher) max_open_trades. |
e39f769
to
09659da
Compare
Objective function, returns smaller number for more optimal results. | ||
Uses Sortino Ratio calculation. | ||
""" | ||
hloc = kwargs["processed"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How can you access the "processed" data? In my loss function, they are not in the kwargs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i don't think this is a working example as is ...
following the PR history, it's an example to show how the data provided via the function below can / could be used.
933aea7
to
5aaa05f
Compare
Summary
This adds a function that should be able to calculate the balance of all open trades on each candle, using close prices.
However results don't match with the function in the issue :)
closes #3020
Quick changelog