c# - Entity Framework Core: `SqlNullValueException: Data is Null.` How to troubleshoot?

Question

I am using Entity Framework Core in an ASP.NET Core application and Controller action and I haven't changed something to the working code nor to the database but I can't tell what is the query performed by Entity Framework Core.

The controller action:

[HttpGet]
// GET: Administration/Companies
public async Task<ActionResult> Index()
{
    var users = await UserManager.Users.ToListAsync();

    var companyEditVMs = await DB.Companies
    .OrderBy(company => company.CompanyId == 1 
        ? "_" + company.CompanyName 
        : company.CompanyName
    )
    Select(a => new CompanyEditVM(HttpContext, a, users.Where(b => b.CompanyId == a.CompanyId)))
    .ToListAsync();

    return View(companyEditVMs);
}

The trace

SqlNullValueException: Data is Null. This method or property cannot be called on Null values.
System.Data.SqlClient.SqlBuffer.get_String()
System.Data.SqlClient.SqlDataReader.GetString(int i)
lambda_method(Closure , DbDataReader )
Microsoft.EntityFrameworkCore.Storage.Internal.TypedRelationalValueBufferFactory.Create(DbDataReader dataReader)
Microsoft.EntityFrameworkCore.Query.Internal.AsyncQueryingEnumerable<T>+AsyncEnumerator.BufferlessMoveNext(DbContext _, bool buffer, CancellationToken cancellationToken)
Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.ExecuteAsync<TState, TResult>(TState state, Func<DbContext, TState, CancellationToken, Task<TResult>> operation, Func<DbContext, TState, CancellationToken, Task<ExecutionResult<TResult>>> verifySucceeded, CancellationToken cancellationToken)
Microsoft.EntityFrameworkCore.Query.Internal.AsyncQueryingEnumerable<T>+AsyncEnumerator.MoveNext(CancellationToken cancellationToken)
Microsoft.EntityFrameworkCore.Query.Internal.AsyncLinqOperatorProvider+ExceptionInterceptor<T>+EnumeratorExceptionInterceptor.MoveNext(CancellationToken cancellationToken)
System.Linq.AsyncEnumerable.Aggregate_<TSource, TAccumulate, TResult>(IAsyncEnumerable<TSource> source, TAccumulate seed, Func<TAccumulate, TSource, TAccumulate> accumulator, Func<TAccumulate, TResult> resultSelector, CancellationToken cancellationToken) in Aggregate.cs
KYC.Core.Areas.Commercial.Controllers.CompaniesController.Index() in CompaniesController.cs
-
        [HttpGet]
        // GET: Administration/Companies
        public async Task<ActionResult> Index()
        {
            var users = await UserManager.Users.ToListAsync();

            var companyEditVMs = await DB.Companies
                .OrderBy(company => company.CompanyId == 1 
                    ? "_" + company.CompanyName 
                    : company.CompanyName
                )
                .Select(a => new CompanyEditVM(HttpContext, a, users.Where(b => b.CompanyId == a.CompanyId)))
                .ToListAsync();
lambda_method(Closure , object )
Microsoft.Extensions.Internal.ObjectMethodExecutorAwaitable+Awaiter.GetResult()
Microsoft.AspNetCore.Mvc.Internal.ActionMethodExecutor+TaskOfActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, object controller, object[] arguments)
System.Threading.Tasks.ValueTask<TResult>.get_Result()

I even tried to do just var companies = await DB.Companies.ToListAsync(). and I have exactly the same exception.

Maybe I would like to be able to fetch the query performed by EF Core to do that manually so that I can try to figure out what's wrong with the query.

I am wondering what possibly could have happened. Especially since other entities like user or countries can still be fetched from the database.

Any idea how can I troubleshoot the actual underlying issue?

[EDIT]

The only that actually changed in the "code" are the nuget references, I basically upgraded pretty much everything although it didn't break a reference in the code it seems it change how EF Core is somehow (wild guess) interpreting the database.

I did reset my git repository to the point just before the nuget packages update happened and things work.,. I then decided to just update EntityFrameworkCore from 2.0.2 to 2.2.4 (also tried with 2.2.0 and ended up with the same result) and the issue happens again... Not sure what have changed between 2.0.2 and 2.2.0 to trigger this exception (but the model is the same just the EF Core version changed)...

Here is the entity definition, it seems to have been auto-generated with a tool from the database table / schema:

    [Table("T_Companies")]
    public partial class Company : INotifyPropertyChanging, INotifyPropertyChanged
    {
        public override string ToString()
        {
            return CompanyId + " " + CompanyName;
        }

        private static PropertyChangingEventArgs emptyChangingEventArgs = new PropertyChangingEventArgs(string.Empty);

        private int _CompanyId;

        private string _CompanyName;

        private int _CompanyTypeId;

        private int _CountryId;

        private string _CompanyVatNumber;

        private string _CompanyStreetAddress;

        private string _CompanyZipCode;

        private string _CompanyCity;

        private string _ContactLastName;

        private string _ContactFirstName;

        private bool? _Active;

        private int? _AccountId;

        private string _CallbackSalt;

        private int? _UserSpaceId;

        private string _Login;

        private string _Pwd;

        private bool _IsTechnicalAccount;

        private DateTime? _StatusDate;

        private int _BankStatusCode;

        private string _PivotalAccount;

        private CompanyType _CompanyType;

        private Country _Country;

        private bool _IsKycIdSent;

        #region Extensibility Method Definitions
        partial void OnLoaded();
        partial void OnCreated();
        partial void OnCompanyIdChanging(int value);
        partial void OnCompanyIdChanged();
        partial void OnCompanyNameChanging(string value);
        partial void OnCompanyNameChanged();

        partial void OnCompanyCityChanging(string value);
        partial void OnCompanyCityChanged();
        partial void OnCompanyZipCodeChanging(string value);
        partial void OnCompanyZipCodeChanged();
        partial void OnContactLastNameChanging(string value);
        partial void OnContactLastNameChanged();
        partial void OnActiveChanging(bool? value);
        partial void OnActiveChanged();
        partial void OnCompanyTypeIdChanging(int value);
        partial void OnCompanyTypeIdChanged();
        partial void OnCountryIdChanging(int value);
        partial void OnCountryIdChanged();
        partial void OnContactFirstNameChanging(string value);
        partial void OnContactFirstNameChanged();
        partial void OnCompanyVatNumberChanging(string value);
        partial void OnCompanyVatNumberChanged();
        partial void OnCompanyStreetAddressChanged();
        partial void OnCompanyStreetAddressChanging(string value);
        partial void OnAccountIdChanging(int? value);
        partial void OnAccountIdChanged();
        partial void OnCallbackSaltChanging(string value);
        partial void OnCallbackSaltChanged();
        partial void OnUserSpaceIdChanging(int? value);
        partial void OnUserSpaceIdChanged();
        partial void OnLoginChanging(string value);
        partial void OnLoginChanged();
        partial void OnPwdChanging(string value);
        partial void OnPwdChanged();
        partial void OnIsTechnicalAccountChanging(bool value);
        partial void OnIsTechnicalAccountChanged();
        partial void OnStatusDateChanging(DateTime? value);
        partial void OnStatusDateChanged();
        partial void OnBankStatusCodeChanging(int value);
        partial void OnBankStatusCodeChanged();
        partial void OnPivotalAccountChanging(string value);
        partial void OnPivotalAccountChanged();
        partial void OnIsKycIdSentChanging(bool value);
        partial void OnIsKycIdSentChanged();

        #endregion

        public Company()
        {
            OnCreated();
        }

        [Key, Column("CompanyId"), DatabaseGenerated(DatabaseGeneratedOption.Identity)]
        public int CompanyId
        {
            get => _CompanyId;
            set
            {
                if (_CompanyId != value)
                {
                    OnCompanyIdChanging(value);
                    SendPropertyChanging();
                    _CompanyId = value;
                    SendPropertyChanged("CompanyId");
                    OnCompanyIdChanged();
                }
            }
        }

        [DataType(DataType.Text), StringLength(1024), Column("CompanyName"), Required]
        public string CompanyName
        {
            get => _CompanyName;
            set
            {
                if (_CompanyName != value)
                {
                    OnCompanyNameChanging(value);
                    SendPropertyChanging();
                    _CompanyName = value;
                    SendPropertyChanged("CompanyName");
                    OnCompanyNameChanged();
                }
            }
        }

        [Column("CompanyTypeId"), Required]
        public int CompanyTypeId
        {
            get => _CompanyTypeId;
            set
            {
                if (_CompanyTypeId != value)
                {
                    OnCompanyTypeIdChanging(value);
                    SendPropertyChanging();
                    _CompanyTypeId = value;
                    SendPropertyChanged("CompanyTypeId");
                    OnCompanyTypeIdChanged();
                }
            }
        }


        [Column("CountryId"), Required]
        public int CountryId
        {
            get => _CountryId;
            set
            {
                if (CountryId != value)
                {
                    OnCountryIdChanging(value);
                    SendPropertyChanging();
                    _CountryId = value;
                    SendPropertyChanged("CountryId");
                    OnCountryIdChanged();
                }
            }
        }


        [DataType(DataType.Text), StringLength(100), Column("CompanyCity"), Required]
        public string CompanyCity
        {
            get => _CompanyCity;
            set
            {
                if (_CompanyCity != value)
                {
                    OnCompanyCityChanging(value);
                    SendPropertyChanging();
                    _CompanyCity = value;
                    SendPropertyChanged("CompanyCity");
                    OnCompanyCityChanged();
                }
            }
        }

        [DataType(DataType.Text), StringLength(100), Column("CompanyStreetAddress"), Required]
        public string CompanyStreetAddress
        {
            get => _CompanyStreetAddress;
            set
            {
                if (_CompanyStreetAddress != value)
                {
                    OnCompanyStreetAddressChanging(value);
                    SendPropertyChanging();
                    _CompanyStreetAddress = value;
                    SendPropertyChanged("CompanyStreetAddress");
                    OnCompanyStreetAddressChanged();
                }
            }
        }

        [DataType(DataType.Text), StringLength(30), Column("CompanyVatNumber"), Required]
        public string CompanyVatNumber
 

Answer 1

The error message indicates that EF Core is trying to read string value for a required property, i.e. a property which should never has null value in the database, but instead the underlying data reader reports null value for that property in some record{s).

Looking at your entity model and corresponding database table, you can see the obvious discrepancy for many string properties -> varchar columns. CompanyStreetAddress, CompanyCity, CompanyZipCode, CompanyVatNumber, ContactFirstName, ContactLastName - all these are marked as [Required] in the model, bit have no corresponding not null constraint in the table.

So the problem is caused by one or more of these columns.

You need to fix that discrepancy - probably by removing [Required] attribute because the constraint is already broken in the existing data.

The fact that it "works" in some older EF Core version doesn't matter - that's incorrect mapping and as such should be fixed. Technically it shouldn't worked from the beginning. But remember that EF Core is still in active development and has many bugs which are fixed in the next release(s). Most likely some code change was made between "working" and "non working" EF Core version which fixes the previous incorrect behavior.