Over the last few years, the financial markets have been regarded as complex and nonlinear dynamic systems. A series of studies has found that many financial market time series display scaling laws and long-range dependence. Therefore, it has been proposed that the Brownian motion in the classical Black-Scholes (BS) model  should be replaced by a process with long-range dependence.
Nowadays, the BS model is the one most commonly used for analyzing financial data, and some scholars have presented modified forms of the BS model which have influential and significant outcomes on option pricing. However, they are still theoretical adaptations and not necessarily consistent with the empirical features of financial return series, such as nonnormality, long-range dependence, etc. For example, some scholars      have showed that returns are of long-range (or short-range) dependence, which suggests strong time-correlations between different events at different time scales    . In the search for better models for describing long-range dependence in financial return series, a mixed fractional Brownian (MFBM) model has been proposed as an improvement of the classical BS model  -  . The advantage of using the MFBM is that the markets are free of arbitrage.
Moreover, Cheridito  has proved that, for , the MFBM is
equivalent to one with Brownian motion, and hence time-step and long-range dependence in return series have no impact on option pricing in a complete financial market without transaction costs. In addition, a number of empirical studies show that the paths of asset prices are discontinuous and that there are jumps in asset prices, both in the stock market and foreign exchange      .
The above researches have an important implication for option pricing. Merton  created a revolution in option pricing when the underlying asset was governed by a diffusion process. Based on this theory, Kou  , Cont and Tankov  also considered the problems of pricing options under a jump diffusion environment in a larger setting. In this paper, to capture jumps or discontinuities, fluctuations and to take into account the long memory property of financial markets, a mixed fractional version of the Merton model is introduced, which is based on a combination of Poisson jumps and MFBM. The mixed fractional Merton (MFM) model is based on the assumption that the underlying asset price is generated by a two-part stochastic process: 1) small, continuous price movements are generated by an MFBM, and 2) large, infrequent price jumps are generated by a Poisson process. This two-part process is intuitively appealing, as it is consistent with an impressive market in which major information arrives infrequently and randomly. This process may provide a description for empirically observed distributions of exchange rate changes that are skewed, leptokurtic, have long memory and fatter tails than comparable normal distributions and apparent nonstationary variance. Further, we will show the impact of the time-step and long-range dependence in return series exactly on option pricing, regardless of whether proportional transaction costs are considered or not in a discrete time setting.
Leland  is a pioneer scholar, who investigated option replication where transaction costs exist in a discrete time setting. In this view, the arbitrage-free arguments presented by Black and Scholes  are not applicable in a model where transaction costs occur at all moments of trading of the stock or bond. The problem is that perfect replication incurs an infinite number of transaction costs because of the infinite variation which exists in the geometric Brownian motion. In this regard, a delta hedge strategy is constructed in accordance with revision conducted a discrete number of times. Transaction costs lead to the failure of the no arbitrage principle and the continuous time trade in general: instead of no arbitrage, the principle of hedge pricing, according to which the price of an option is defined as the minimum level of initial wealth needed to hedge the option, comes into force.
According to the empirical findings obtained before and the views of behavioral finance and econophysics, we are motivated to examine the problem that exists in option pricing, while the dynamics of price follows a mixed fractional jump-diffusion process under the transaction costs. We assume that satisfies
where and are fixed; is a Brownian motion; is a
fractional Brownian motion with Hurst parameter ; is a Poisson
process with intensity ; and J is a positive random variable. We assume that and J are independent.
This paper is organized into several sections. In Section 2, we will study the problem of option pricing with transaction costs by applying delta hedging strategy. In addition, a new framework for pricing European option is obtained when the stock price is satisfied in Equation (1.1). Section 3 is devoted to empirical studies and simulations to show the performance of the MFM model. A conclusion is presented in Section 4.
2. Pricing Option by Mixed Fractional Version of Merton Model with Transaction Costs
Suppose be a standard Brownian motion and be a fractional Brownian motion with the Hurst parameter , both defined
on complete probability space , the absolute price jump size J is a nonnegative random variable drawn from lognormal distribution, i.e. , which implies
and a Poisson process with rate . Additionally, the processes and J are independent, P is the real world probability measure and denotes the P-augmentation of filtration generated by .
The objective of this section is to derive a stock pricing formula under transaction costs in a discrete time setting. Consider -market with a bond and a stock , where
The groundwork of modeling the effects of transaction costs was done by Leland  . He adopted the hedging strategy of rehedging at every time-step . That is, with every the portfolio is rebalanced, whether or not this is optimal in any sense. In the following proportional transaction cost option pricing model, we follow the other usual assumptions in the Black-Scholes model, but with the following exceptions:
1) The price of the underlying stock at time t satisfies Equation (2.2).
2) The portfolio is revised every where is a finite and fixed, small time-step.
3) Transaction costs are proportional to the value of the transaction in the underlying. Let k denote the round trip transaction cost per unit dollar of transaction. Suppose shares are bought or sold at the price , then the transaction cost is given by in either buying or selling, where k is a constant. The value of k will depend on the individual investor. In the MFM model, where transaction costs are incurred at every time the stock or the bond is traded, the no arbitrage argument used by Black and Scholes no longer applies. The problem is that due to the infinite variation of the MFBM, perfect replication incurs an infinite amount of transaction costs.
4) The hedge portfolio has an expected return equal to that from an option. This is exactly the same valuation policy as earlier on discrete hedging with no transaction costs.
5) Traditional economics assumes that traders are rational and maximize their utility. However, if their behaviour is assumed to be boundedly rational, the traders' decisions can be explained both by their reaction to the past stock price, according to a standard speculative behaviour, and by imitation of other traders’ past decisions, according to common evidence in social psychology. It is well known that the delta-hedging strategy plays a central role in the theory of option pricing and that it is popularly used on the trading floor. Therefore, based on the availability heuristic, suggested by Tversky and Kahneman  , traders are assumed to follow, anchor, and imitate the Black-Scholes delta-hedging strategy to price an option. In this case, delta-hedging argument is a partial and imperfect hedging strategy, which does not eliminate all of the risk. However, as mentioned in the paper  , in most models of stock fluctuations, except for very special cases, risk in option trading cannot be eliminated and strict arbitrage opportunities do not exist, whatever be the price of the option. The risk cannot be eliminated is furthermore the fundamental reason for the very existence of option markets.
Delta hedging is an options strategy that aims to reduce, or hedge, the risk associated with price movements in the underlying asset, by offsetting long and short positions. For example, a long call position may be delta hedged by shorting the underlying stock. This strategy is based on the change in premium, or price of option, caused by a change in the price of the underlying security. In this section we use the delta hedging strategy to obtain a pricing formula for European call option.
Let the price of European call option be denoted with expiration T and strike price K by with boundary conditions:
Then, is derived by the following theorem.
Theorem 2.1. The price at every of a European call option with strike price K that matures at time T is given by
Moreover, satisfies the following equation
is the signum function of ; n is the number of prices jumps;
is a small and fixed time-step; k is the transaction costs and is the cumulative normal distribution.
Moreover, using the put call parity, we can easily obtain the valuation model for a put currency option, which is provided by the following corollary.
Corollary 2.1. The value of European put option with transaction costs is given by
3. Properties of Pricing Formula
In this section, we present the properties of MFM’s log-return density. The effects of Hurst parameter and time-step on our modified volatility are also discussed in the discrete time and continuous time cases. Then we show that these parameters play a significant role in a discrete time setting, both with and without transaction costs.
3.1. Log-Return Density
In the case of MFM the log return jump size is assumed to be and the probability density of log return is achieved as a quickly converging series of the following form:
The term is the probability that the asset price jumps
n times during the time interval of length t. And is the mixed fractional normal density of log-return. It supposes that the asset price jumps i times in the time interval of t. As a result, in the MFM model, the log-return density can be described as the weighted average of the mixed fractional normal density by the probability that the asset price jumps n times.
The outstanding properties of log-return density are observed in the MFM. Firstly, the sign refers to the expected log-return jump size, , which indicates the skewness sign. If , the log-return density shows negatively skewed, and if , it is symmetric as displayed in Figure 1 (Table 1).
Secondly, larger value of intensity (which means that jumps are expected to occur more frequently) makes the density fatter-tailed as illustrated in Figure 2. Note that the excess kurtosis in the case is much smaller than in the case or . This is because excess kurtosis is a standardized measure (by standard deviation) (Table 2).
3.2. The Impact of Parameters
Mantegna and Stanley  as pioneer scholars proposed the scaling invariance method from the complex science of economic systems which led to numerous investigations into scaling laws in finance. The major question in economics is whether the price impact of scaling law and long-range dependence is significant in option pricing. The answer to this question is assured. For instance, one of the significant issues in finance concerning the modeling of high-frequency data is related to analyzing the volatility in different time scales.
Figure 1. MFM’s log-return density. Fixed parameters are , , , , , , and .
Figure 2. MFM’s log-return density. Fixed parameters are , , , , , , and .
Table 1. Annualized moments of Merton’s log-return density in Figure 1.
Table 2. Annualized moments of Merton’s log-return density in Figure 2.
Remark 3.1. In a continuous time setting without transaction costs the implied volatility is , thus the option value is similar to
the Merton jump diffusion model  . Moreover, if in the absence of transaction costs and jump case, the MFM model reduces to the BS model
which shows that the Hurst parameter H and time-step have no effect on option pricing model in a continuous time setting .
Remark 3.2. In a discrete time setting without transaction costs , if jump occurs, the modified volatility is
and when jump does not occur , from Equation (2.5), we have
which demonstrates that the delta hedging strategy in a discrete time case is fundamentally different in comparison with a continuous time case. It also indicates that the scaling exponent and time-step play a significant role in option pricing theory. Figure 3 illustrates the impacts of the time-step, Hurst parameter, mean jump, and jump intensity on our option pricing model.
Remark 3.3. From  we infer there exists such that
where , k is small enough
Figure 3. Modified volatility. Fixed parameters are , , , , , , and .
as k is small enough.
Hence, there exists a such that holds.
Then the minimal price of an option with respect to transaction costs is displayed as with in Equation (2.4). can be applied to the real price of an option.
To capture the long memory and discontinuous property, this article focuses on the problem of pricing European option in a mixed fractional Merton environment without using the arbitrage argument. We obtain a mixed fractional version of Merton model for pricing European option with transaction costs. Some properties of mixed fractional Merton’s log-return density are discussed. Moreover, we derive that the Hurst parameter H and time-step play a significant role in pricing option in a discrete time setting for cases both with and without transaction costs. But these parameters have no impact on option pricing in a continuous time setting.
Proof of Theorem 2.1. We consider a replicating portfolio with units of financial underlying asset and one unit of the option. Then, the value of the portfolio at time t is
Now, the movement in and is considered under discrete time interval . In view of this, we suppose that trading takes place at the specific time points of t and . It can be said that the number of shares through the use of delta-hedging strategy and the present stock price are constantly held during the rebalancing interval . Then, the movement in the value of the portfolio after time interval is defined as follows:
where is the movement of the underlying stock price, is the movement of the number of units of stock held in the portfolio, and is the change in the value of the portfolio.
Since the time-step and the asset change are both small, according to Taylor’s formulae we have if with probability , so
Since and are continuous, then from  we have
Thus, we can get
If with probability and the jump of in is assumed to occur at current time t, then
Based on the above assumptions iv and v, we have, i.e.. Then
where the current stock price is given. Since
and, from Equations (4.1) - (4.17), we can get
Hence, we assume that
Note that the term is nonlinear, except when does not change sign for all. Since it represents the degree of
mishedging of the portfolio, it is not surprising to observe that is involved in the transaction cost term. We may rewrite Equation (4.19) in the form which resembles the Merton equation:
where and the implied volatility is given by
If, Equation (4.20) becomes mathematically ill-posed. This occurs when and. However, it is known that is always positive for the simple European call and put options in the absence of transaction costs. If we postulate the same sign behaviour for in the presence of transaction costs, Equation (4.20) becomes linear under such an assumption so that the Merton formula becomes applicable except that the modified volatility should be used as the volatility parameter. Moreover, if from Equation (4.20) we obtain
is the signum function of, is a small and fixed time-step, k is the transaction costs and is the cumulative normal distribution.