Description
Let's imagine scenario:
We are running in prod, doing daily operations. We get some garbage from source and during automatic processing, we fail to process one identity (most probably in model). We are running in task, configured to skip wrong records and continue with next one.
We finish the task and operation person has to analyze it and report to HR to correct the record.
Where can we find at least basic information what record (ID,username) was malformed (ideal what data were wrong).
We have hit this scenario when one attribute value was too long, but we can do other validations and fail to process.
But we were not able to find a record at least with username or some other ID not in Tasks UI tabs nor Audit record.
Technical details for our scenario:
2019-08-14 08:30:48,603 [] [midPointScheduler_Worker-8] ERROR (com.evolveum.midpoint.provisioning.impl.ShadowCache): Exception during processing object as part of live sync
hronization in Task(id:1565698122902-0-1, name:Kafka 2, oid:892604e3-8b40-44a6-b216-6cf266a0dd14).
com.evolveum.midpoint.util.exception.SystemException: org.hibernate.exception.DataException: could not execute batch
at com.evolveum.midpoint.repo.sql.helpers.BaseHelper.handleGeneralRuntimeException(BaseHelper.java:173)
at com.evolveum.midpoint.repo.sql.helpers.BaseHelper.handleGeneralException(BaseHelper.java:153)
at com.evolveum.midpoint.repo.sql.helpers.ObjectUpdater.addObjectAttempt(ObjectUpdater.java:148)
at com.evolveum.midpoint.repo.sql.SqlRepositoryServiceImpl.addObject(SqlRepositoryServiceImpl.java:473)
at com.evolveum.midpoint.repo.cache.RepositoryCache.addObject(RepositoryCache.java:455)
at com.evolveum.midpoint.provisioning.impl.ShadowManager.findOrAddShadowFromChange(ShadowManager.java:493)
at com.evolveum.midpoint.provisioning.impl.ShadowCache.preProcessChange(ShadowCache.java:2899)
at com.evolveum.midpoint.provisioning.impl.ShadowCache.processSynchronization(ShadowCache.java:2671)
at com.evolveum.midpoint.provisioning.impl.ShadowCache.synchronize(ShadowCache.java:2547)
at com.evolveum.midpoint.provisioning.impl.ProvisioningServiceImpl.synchronize(ProvisioningServiceImpl.java:364)
at com.evolveum.midpoint.model.impl.sync.LiveSyncTaskHandler.run(LiveSyncTaskHandler.java:114)
at com.evolveum.midpoint.task.quartzimpl.execution.HandlerExecutor.executePlainTaskHandler(HandlerExecutor.java:71)
at com.evolveum.midpoint.task.quartzimpl.execution.HandlerExecutor.executeHandler(HandlerExecutor.java:61)
at com.evolveum.midpoint.task.quartzimpl.execution.JobExecutor.executeHandler(JobExecutor.java:652)
at com.evolveum.midpoint.task.quartzimpl.execution.JobExecutor.executeSingleTask(JobExecutor.java:471)
at com.evolveum.midpoint.task.quartzimpl.execution.JobExecutor.execute(JobExecutor.java:187)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:588)
Caused by: javax.persistence.PersistenceException: org.hibernate.exception.DataException: could not execute batch
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:149)
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:157)
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:164)
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1443)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:493)
at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3207)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2413)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:473)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordin
atorImpl.java:156)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$100(JdbcResourceLocalTransactionCoordinatorImpl.java:
38)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransac
tionCoordinatorImpl.java:231)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:68)
at com.evolveum.midpoint.repo.sql.helpers.ObjectUpdater.addObjectAttempt(ObjectUpdater.java:140)
... 15 common frames omitted
Caused by: org.hibernate.exception.DataException: could not execute batch
at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:118)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:111)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:121)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.doExecuteBatch(BatchingBatch.java:97)
at org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl.execute(AbstractBatchImpl.java:147)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.executeBatch(JdbcCoordinatorImpl.java:212)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:618)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:463)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1437)
... 24 common frames omitted
Caused by: java.sql.BatchUpdateException: (conn=26546) Data too long for column 'stringValue' at row 1
at org.mariadb.jdbc.MariaDbStatement.executeBatchExceptionEpilogue(MariaDbStatement.java:289)
at org.mariadb.jdbc.MariaDbPreparedStatementClient.executeBatch(MariaDbPreparedStatementClient.java:306)
at com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:128)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:111)
... 32 common frames omitted
Caused by: java.sql.SQLDataException: (conn=26546) Data too long for column 'stringValue' at row 1
at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.get(ExceptionMapper.java:225)
at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.getException(ExceptionMapper.java:165)
at org.mariadb.jdbc.MariaDbStatement.executeBatchExceptionEpilogue(MariaDbStatement.java:286)
... 36 common frames omitted
Caused by: java.sql.SQLException: Data too long for column 'stringValue' at row 1
Query is: insert into m_object_ext_string (item_id, owner_oid, ownerType, stringValue) values (118, '7e0035aa-d6e3-4b9c-8141-ff177b7690d9', 1, 'some veeeery long value')
java thread: MariaDb-bulk-4
at org.mariadb.jdbc.internal.util.LogQueryTool.exceptionWithQuery(LogQueryTool.java:126)
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol$1.handleResultException(AbstractQueryProtocol.java:625)
at org.mariadb.jdbc.internal.protocol.AsyncMultiRead.call(AsyncMultiRead.java:142)
at org.mariadb.jdbc.internal.protocol.AsyncMultiRead.call(AsyncMultiRead.java:66)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Attachments
Issue Links
- relates to
-
MID-5801 Livesync from CSV connector resource has bad conflict user processing, fatal error is not recorded to shadow
-
- Closed
-