You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-03 - 00:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sat May 02, 2026 12:45:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
792903997920,44cyclictest12-21ksoftirqd/011:25:220
792903997920,44cyclictest12-21ksoftirqd/011:25:220
79290399745,26cyclictest131rcu_preempt10:15:540
79290399726,7cyclictest121ksoftirqd/011:21:300
79291799718,9cyclictest37-21ksoftirqd/310:30:013
792903997117,49cyclictest12-21ksoftirqd/012:37:030
79290399703,50cyclictest825185-21ssh10:10:030
792903997015,49cyclictest131rcu_preempt08:00:000
792903996913,49cyclictest131rcu_preempt10:45:020
792903996820,44cyclictest12-21ksoftirqd/009:40:110
792903996518,42cyclictest12-21ksoftirqd/010:39:040
792903996513,5cyclictest131rcu_preempt12:20:000
792903996424,36cyclictest12-21ksoftirqd/009:33:340
792903996316,32cyclictest12-21ksoftirqd/010:15:020
792903996316,32cyclictest12-21ksoftirqd/010:15:020
792903996211,5cyclictest131rcu_preempt08:20:000
79291799617,12cyclictest37-21ksoftirqd/311:32:043
792917996137,15cyclictest3467602-21systemd-journal07:40:003
792917996137,15cyclictest3467602-21systemd-journal07:40:003
792917996116,6cyclictest825022-21kworker/u8:210:15:023
792917996116,6cyclictest825022-21kworker/u8:210:15:023
792903996110,45cyclictest0-21swapper/010:25:020
79291799607,9cyclictest37-21ksoftirqd/312:40:013
792917996042,13cyclictest1-21systemd08:00:003
79290399608,6cyclictest131rcu_preempt12:05:020
792903996019,7cyclictest131rcu_preempt12:30:010
792903996019,10cyclictest131rcu_preempt07:40:000
792903996019,10cyclictest131rcu_preempt07:40:000
79291799598,11cyclictest37-21ksoftirqd/307:35:013
79291799594,13cyclictest37-21ksoftirqd/309:50:183
792917995912,12cyclictest37-21ksoftirqd/309:25:263
792903995928,14cyclictest799389-21ssh09:10:250
792903995920,34cyclictest16850irq/134-800000012:15:010
792903995919,5cyclictest131rcu_preempt11:07:520
792903995919,5cyclictest131rcu_preempt11:07:510
792903995918,6cyclictest131rcu_preempt12:21:540
792903995918,34cyclictest275-21dbus-daemon10:05:010
792903995911,34cyclictest131rcu_preempt10:50:010
792903995911,34cyclictest131rcu_preempt10:50:000
792917995818,10cyclictest37-21ksoftirqd/311:18:043
792903995816,37cyclictest12-21ksoftirqd/011:39:530
792903995814,40cyclictest12-21ksoftirqd/010:58:430
792903995814,40cyclictest12-21ksoftirqd/010:58:430
792903995810,42cyclictest131rcu_preempt09:26:440
792917995711,8cyclictest37-21ksoftirqd/310:43:263
792917995711,11cyclictest37-21ksoftirqd/308:15:013
792917995710,7cyclictest37-21ksoftirqd/311:05:013
792907995711,6cyclictest131rcu_preempt09:15:001
79290399579,43cyclictest131rcu_preempt09:50:020
79290399579,43cyclictest131rcu_preempt09:50:020
792903995712,40cyclictest12-21ksoftirqd/011:00:440
792917995647,7cyclictest37-21ksoftirqd/307:50:013
792903995626,14cyclictest853815-21ssh11:16:290
792903995610,12cyclictest131rcu_preempt09:22:460
79291799556,10cyclictest37-21ksoftirqd/312:00:013
79291799553,14cyclictest37-21ksoftirqd/311:15:013
792917995511,8cyclictest37-21ksoftirqd/309:25:023
792910995545,6cyclictest797567-21kworker/2:109:19:272
79290399558,5cyclictest131rcu_preempt11:15:000
79290399556,10cyclictest12-21ksoftirqd/011:52:160
792903995520,31cyclictest16850irq/134-800000009:55:290
792903995520,31cyclictest16850irq/134-800000009:55:290
792903995517,34cyclictest12-21ksoftirqd/012:33:540
792903995516,10cyclictest131rcu_preempt12:05:020
792903995512,9cyclictest12-21ksoftirqd/011:32:090
79291799549,8cyclictest37-21ksoftirqd/308:20:003
79291799549,14cyclictest37-21ksoftirqd/312:25:473
79291799548,12cyclictest37-21ksoftirqd/312:12:513
79291799545,10cyclictest37-21ksoftirqd/310:40:003
792917995434,14cyclictest1-21systemd11:45:013
792917995411,9cyclictest37-21ksoftirqd/308:25:023
792917995411,9cyclictest37-21ksoftirqd/308:25:023
792917995410,6cyclictest37-21ksoftirqd/312:25:013
79291099542,26cyclictest833844-21ssh10:30:022
792907995427,11cyclictest23-21ksoftirqd/111:20:011
79290399547,8cyclictest12-21ksoftirqd/012:00:030
79290399547,6cyclictest131rcu_preempt10:31:070
79290399544,6cyclictest131rcu_preempt11:50:000
792903995418,32cyclictest16850irq/134-800000011:41:130
792903995415,10cyclictest12-21ksoftirqd/008:45:020
79291799539,8cyclictest37-21ksoftirqd/307:30:013
79291799537,10cyclictest37-21ksoftirqd/310:49:553
79291799537,10cyclictest37-21ksoftirqd/310:49:553
79291799536,6cyclictest37-21ksoftirqd/310:56:173
79291799536,6cyclictest37-21ksoftirqd/310:56:163
79291799532,14cyclictest862409-21ssh11:36:083
792917995315,11cyclictest37-21ksoftirqd/310:22:553
792910995344,5cyclictest832072-21kworker/2:211:45:092
792910995343,6cyclictest797567-21kworker/2:109:04:542
79290799539,6cyclictest131rcu_preempt10:30:431
79290799532,37cyclictest864043-21ssh11:40:031
79290399539,9cyclictest12-21ksoftirqd/010:51:510
79290399539,7cyclictest12-21ksoftirqd/008:30:010
792903995319,30cyclictest16850irq/134-800000009:35:510
792903995319,30cyclictest16850irq/134-800000009:35:510
792903995318,6cyclictest16850irq/134-800000010:28:300
792903995312,6cyclictest131rcu_preempt07:55:010
792903995310,5cyclictest131rcu_preempt09:19:190
79291799529,10cyclictest37-21ksoftirqd/311:55:023
79291799527,10cyclictest37-21ksoftirqd/309:00:003
79291799521,8cyclictest0-21swapper/309:35:023
79291799521,8cyclictest0-21swapper/309:35:023
792910995244,5cyclictest832072-21kworker/2:211:05:012
792910995244,5cyclictest798591-21kworker/2:009:52:352
792910995243,6cyclictest798591-21kworker/2:009:35:202
792910995243,6cyclictest798591-21kworker/2:009:35:202
792910995243,5cyclictest832072-21kworker/2:210:54:192
792910995243,5cyclictest797567-21kworker/2:110:20:022
792910995241,6cyclictest838012-21kworker/2:311:53:402
79290799524,9cyclictest23-21ksoftirqd/111:30:521
79290399524,12cyclictest12-21ksoftirqd/008:55:000
792903995217,18cyclictest818759-21ssh09:55:020
792903995211,8cyclictest131rcu_preempt09:05:000
79291799518,8cyclictest37-21ksoftirqd/312:18:233
79291799517,8cyclictest37-21ksoftirqd/312:32:333
79291799515,8cyclictest37-21ksoftirqd/310:31:523
79291799515,11cyclictest37-21ksoftirqd/309:31:243
79291799514,10cyclictest37-21ksoftirqd/310:16:413
79291799511,9cyclictest37-21ksoftirqd/311:09:443
79291799511,9cyclictest37-21ksoftirqd/311:09:443
792917995112,10cyclictest37-21ksoftirqd/307:45:023
79291799511,14cyclictest37-21ksoftirqd/309:56:163
79291799511,14cyclictest37-21ksoftirqd/309:56:153
792917995110,10cyclictest37-21ksoftirqd/311:49:353
792910995143,5cyclictest838012-21kworker/2:312:27:242
792910995143,5cyclictest838012-21kworker/2:312:05:282
792910995143,5cyclictest832072-21kworker/2:211:20:012
792910995142,5cyclictest838012-21kworker/2:311:58:542
792910995141,6cyclictest832072-21kworker/2:211:07:162
792910995141,6cyclictest832072-21kworker/2:211:07:152
792910995141,6cyclictest798591-21kworker/2:009:30:002
792910995132,14cyclictest1-21systemd09:50:012
792910995132,14cyclictest1-21systemd09:50:012
792910995121,7cyclictest800205-21sh09:12:312
79290799518,5cyclictest131rcu_preempt12:30:001
79290799517,5cyclictest131rcu_preempt09:51:081
792907995117,21cyclictest131rcu_preempt12:33:041
79290399519,7cyclictest131rcu_preempt08:10:100
79290399518,5cyclictest131rcu_preempt07:17:530
792903995118,30cyclictest12-21ksoftirqd/008:40:010
792903995118,30cyclictest12-21ksoftirqd/008:40:010
792903995113,5cyclictest131rcu_preempt08:49:540
792903995113,5cyclictest131rcu_preempt08:49:540
792903995112,5cyclictest131rcu_preempt09:00:010
79291799504,8cyclictest37-21ksoftirqd/307:15:023
79291799503,14cyclictest857634-21cat11:25:023
79291799503,11cyclictest37-21ksoftirqd/312:02:073
792910995042,5cyclictest838012-21kworker/2:312:12:582
792910995042,5cyclictest832072-21kworker/2:211:41:552
792910995042,5cyclictest832072-21kworker/2:210:42:132
792910995042,5cyclictest798591-21kworker/2:010:01:072
792910995042,5cyclictest797567-21kworker/2:109:22:382
792907995022,17cyclictest23-21ksoftirqd/109:42:471
79290399503,11cyclictest12-21ksoftirqd/009:05:540
79290399503,11cyclictest12-21ksoftirqd/009:05:540
79291799493,8cyclictest37-21ksoftirqd/309:45:273
79291799493,8cyclictest37-21ksoftirqd/309:45:273
79291799493,10cyclictest37-21ksoftirqd/312:10:023
79291799492,11cyclictest37-21ksoftirqd/310:01:433
792917994913,10cyclictest0-21swapper/309:20:013
79291799491,11cyclictest37-21ksoftirqd/310:10:013
792910994941,5cyclictest838012-21kworker/2:312:39:142
792910994941,5cyclictest832072-21kworker/2:212:21:092
792910994941,5cyclictest832072-21kworker/2:211:33:402
792910994941,5cyclictest797567-21kworker/2:110:12:052
792910994941,5cyclictest797567-21kworker/2:110:12:052
792910994940,5cyclictest798591-21kworker/2:009:35:002
792910994934,12cyclictest3467602-21systemd-journal10:21:552
792910994932,13cyclictest3467602-21systemd-journal07:11:542
792910994932,12cyclictest824985-21cron10:10:002
792907994935,10cyclictest831521-21munin-plugin-st10:25:001
792907994921,19cyclictest0-21swapper/112:08:101
79290799491,8cyclictest0-21swapper/110:52:491
792907994913,5cyclictest131rcu_preempt09:05:001
792903994919,5cyclictest131rcu_preempt08:35:000
79291799485,8cyclictest37-21ksoftirqd/310:51:483
79291799485,10cyclictest37-21ksoftirqd/309:45:013
79291799482,8cyclictest37-21ksoftirqd/309:10:203
792917994821,7cyclictest37-21ksoftirqd/311:27:153
792917994821,7cyclictest37-21ksoftirqd/311:27:153
79291799482,15cyclictest37-21ksoftirqd/307:55:023
792910994841,5cyclictest797567-21kworker/2:110:33:282
792910994840,5cyclictest838012-21kworker/2:312:00:192
792910994840,5cyclictest832072-21kworker/2:211:24:252
792910994840,5cyclictest832072-21kworker/2:210:56:012
792910994840,5cyclictest832072-21kworker/2:210:56:002
792910994840,5cyclictest798591-21kworker/2:009:58:152
792910994840,5cyclictest798591-21kworker/2:009:58:152
792910994840,5cyclictest798591-21kworker/2:009:42:472
792910994839,5cyclictest792497-21kworker/2:308:28:542
792910994834,10cyclictest1-21systemd07:30:012
792910994829,14cyclictest798561-21run-parts09:00:012
79290799489,26cyclictest0-21swapper/110:05:021
79290799486,5cyclictest131rcu_preempt09:57:191
79290799486,5cyclictest131rcu_preempt09:57:191
792907994830,13cyclictest275-21dbus-daemon11:10:011
792907994830,13cyclictest275-21dbus-daemon11:10:011
792907994823,11cyclictest23-21ksoftirqd/109:10:011
792907994823,11cyclictest23-21ksoftirqd/109:10:011
792907994813,8cyclictest131rcu_preempt11:45:021
792907994813,17cyclictest23-21ksoftirqd/112:00:291
792907994813,10cyclictest131rcu_preempt11:00:001
792907994813,10cyclictest131rcu_preempt11:00:001
792907994811,5cyclictest131rcu_preempt12:15:251
79290399483,9cyclictest12-21ksoftirqd/007:15:020
792903994811,5cyclictest131rcu_preempt07:22:310
79290399481,11cyclictest794191-21sshd07:35:010
792903994810,9cyclictest131rcu_preempt07:45:010
792903994810,11cyclictest12-21ksoftirqd/008:05:000
79291799479,8cyclictest37-21ksoftirqd/307:20:023
792917994733,11cyclictest798814-21sshd09:04:553
79291799472,12cyclictest37-21ksoftirqd/308:45:023
792910994739,5cyclictest832072-21kworker/2:212:19:222
792910994739,5cyclictest832072-21kworker/2:210:45:542
792910994739,5cyclictest832072-21kworker/2:210:45:532
792910994733,10cyclictest275-21dbus-daemon11:40:012
79290799476,6cyclictest131rcu_preempt09:20:011
79290799475,5cyclictest131rcu_preempt10:42:461
79290799474,6cyclictest131rcu_preempt11:57:181
79290799472,4cyclictest131rcu_preempt09:32:111
792907994723,9cyclictest23-21ksoftirqd/110:15:391
792907994712,9cyclictest131rcu_preempt11:48:451
792907994711,10cyclictest23-21ksoftirqd/111:03:591
79291799467,9cyclictest37-21ksoftirqd/309:10:023
79291799467,9cyclictest37-21ksoftirqd/309:10:023
79291799465,8cyclictest37-21ksoftirqd/308:40:023
79291799465,8cyclictest37-21ksoftirqd/308:40:023
792910994639,5cyclictest832072-21kworker/2:211:25:302
792910994639,5cyclictest832072-21kworker/2:211:25:302
792910994639,4cyclictest838012-21kworker/2:312:30:322
792910994638,5cyclictest792497-21kworker/2:308:10:442
792910994637,6cyclictest792497-21kworker/2:307:52:122
79290799467,7cyclictest23-21ksoftirqd/109:50:021
79290799467,7cyclictest23-21ksoftirqd/109:50:021
79290799467,6cyclictest131rcu_preempt10:37:001
79290799463,6cyclictest131rcu_preempt09:35:161
79290799463,6cyclictest131rcu_preempt09:35:151
79290799461,9cyclictest0-21swapper/111:21:561
792907994614,13cyclictest23-21ksoftirqd/109:00:001
792907994611,6cyclictest131rcu_preempt10:11:111
792907994611,6cyclictest131rcu_preempt10:11:111
79290399469,6cyclictest131rcu_preempt08:25:000
79290399469,6cyclictest131rcu_preempt08:25:000
792917994534,8cyclictest37-21ksoftirqd/307:20:003
79291099455,22cyclictest796979-21kworker/u8:308:45:022
792910994539,4cyclictest832072-21kworker/2:211:13:372
79290799456,8cyclictest131rcu_preempt12:25:011
79290799456,12cyclictest23-21ksoftirqd/111:54:361
79290799455,7cyclictest23-21ksoftirqd/110:05:001
79290799455,7cyclictest131rcu_preempt10:27:251
79290799455,5cyclictest131rcu_preempt08:15:001
79290799453,6cyclictest131rcu_preempt07:10:011
792907994522,8cyclictest23-21ksoftirqd/111:13:371
79291799444,9cyclictest37-21ksoftirqd/308:50:023
79291799444,9cyclictest37-21ksoftirqd/308:50:023
792917994426,12cyclictest1-21systemd08:55:003
79291799442,10cyclictest795858-21cat08:05:023
79291099444,3cyclictest131rcu_preempt08:05:032
792910994430,10cyclictest798265-21sshd08:50:222
792910994429,11cyclictest30-21ksoftirqd/207:49:542
792910994428,11cyclictest3467602-21systemd-journal07:20:012
792907994428,11cyclictest3467602-21systemd-journal07:30:001
792907994410,6cyclictest131rcu_preempt12:15:011
792903994412,6cyclictest131rcu_preempt08:10:010
79291799437,6cyclictest37-21ksoftirqd/308:30:003
792910994328,12cyclictest796725-21sshd08:20:142
792910994328,12cyclictest796725-21sshd08:20:142
792910994328,11cyclictest275-21dbus-daemon07:25:012
79290799439,8cyclictest131rcu_preempt11:26:021
79290799439,8cyclictest131rcu_preempt11:26:021
79290799437,6cyclictest131rcu_preempt07:20:001
792907994329,8cyclictest23-21ksoftirqd/108:40:011
792907994329,8cyclictest23-21ksoftirqd/108:40:011
79290799432,5cyclictest131rcu_preempt12:38:541
79290799432,5cyclictest131rcu_preempt09:29:231
792907994313,18cyclictest23-21ksoftirqd/109:21:081
79291799426,10cyclictest37-21ksoftirqd/308:35:003
79290799427,4cyclictest131rcu_preempt10:50:001
79290799427,4cyclictest131rcu_preempt10:50:001
79290799425,9cyclictest23-21ksoftirqd/107:55:011
79290799423,11cyclictest131rcu_preempt08:10:011
792917994128,8cyclictest792914-21cat07:10:023
792910994134,5cyclictest832072-21kworker/2:210:38:312
792910994110,7cyclictest131rcu_preempt07:59:592
792907994123,12cyclictest23-21ksoftirqd/107:50:021
79290399419,5cyclictest131rcu_preempt07:50:020
792910994030,6cyclictest797567-21kworker/2:109:10:022
792910994030,6cyclictest797567-21kworker/2:109:10:012
79290799406,6cyclictest131rcu_preempt07:14:581
79290799404,6cyclictest131rcu_preempt08:30:001
792907994013,5cyclictest131rcu_preempt07:25:021
792903994011,6cyclictest131rcu_preempt07:30:010
792910993931,4cyclictest792497-21kworker/2:308:20:012
79291099391,3cyclictest131rcu_preempt08:35:012
79290799399,7cyclictest131rcu_preempt07:40:021
79290799399,7cyclictest131rcu_preempt07:40:021
79290799397,5cyclictest131rcu_preempt08:05:001
79290799393,6cyclictest131rcu_preempt08:35:011
792907993926,9cyclictest797773-21sshd08:43:541
79291099387,11cyclictest0-21swapper/208:40:022
79291099387,11cyclictest0-21swapper/208:40:022
79290799389,4cyclictest131rcu_preempt08:25:011
79290799389,4cyclictest131rcu_preempt08:25:011
79290799384,6cyclictest131rcu_preempt07:42:071
79290799383,5cyclictest131rcu_preempt08:50:021
79290799383,5cyclictest131rcu_preempt08:50:021
792907993826,7cyclictest796607-21cat08:20:021
792907993825,10cyclictest275-21dbus-daemon07:35:011
79291799375,5cyclictest131rcu_preempt08:06:543
792910993724,9cyclictest309-21in:imuxsock07:40:002
792910993724,9cyclictest309-21in:imuxsock07:40:002
79291099371,32cyclictest48-21kcompactd008:05:262
792910993626,7cyclictest794191-21sshd07:35:012
79290799361,5cyclictest131rcu_preempt08:54:551
792907993613,16cyclictest23-21ksoftirqd/107:56:011
792910993524,9cyclictest30-21ksoftirqd/207:45:022
79290399332,5cyclictest131rcu_preempt07:10:020
79291099323,4cyclictest131rcu_preempt08:50:002
79291099323,4cyclictest131rcu_preempt08:50:002
792910992923,4cyclictest792849-21latency_hist07:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional