You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-03 - 13:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sun May 03, 2026 00:45:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
948025998479,3cyclictest0-21swapper/000:35:020
948025998479,3cyclictest0-21swapper/000:35:020
948027997829,16cyclictest1-21systemd21:20:001
948027997431,12cyclictest0-21swapper/122:55:011
94802599745,50cyclictest1042901-21cat00:15:020
94802599745,50cyclictest1042901-21cat00:15:020
948027997014,25cyclictest1035682-40gzip00:00:041
94802599706,32cyclictest12-21ksoftirqd/020:20:030
948025997052,11cyclictest16850irq/134-800000000:10:000
948025997046,5cyclictest16850irq/134-800000022:25:020
948025997046,5cyclictest16850irq/134-800000022:25:010
948027996930,13cyclictest23-21ksoftirqd/100:00:111
94802599698,40cyclictest120-21kswapd023:35:020
94802599692,48cyclictest1016649-21ssh23:20:020
94802599692,48cyclictest1016649-21ssh23:20:020
948025996817,32cyclictest974953-21ssh21:53:160
94802599665,7cyclictest121ksoftirqd/021:45:150
948025996622,30cyclictest12-21ksoftirqd/022:49:390
948027996529,11cyclictest23-21ksoftirqd/121:25:021
948025996521,41cyclictest12-21ksoftirqd/021:13:380
948032996455,5cyclictest947776-21kworker/2:223:45:152
94802599642,44cyclictest980580-21ssh22:05:010
94802599636,7cyclictest12-21ksoftirqd/000:17:580
94802599627,8cyclictest12-21ksoftirqd/000:30:010
94802599625,40cyclictest12-21ksoftirqd/021:05:010
94802599625,12cyclictest12-21ksoftirqd/000:25:020
94802599614,7cyclictest121ksoftirqd/021:30:260
948025996131,25cyclictest12-21ksoftirqd/023:58:540
948027996011,36cyclictest221rcuc/119:50:021
94802599609,11cyclictest12-21ksoftirqd/000:00:090
94802599608,47cyclictest0-21swapper/023:00:000
948025996037,18cyclictest3467602-21systemd-journal22:35:000
948035995918,3cyclictest1042896-21ssh00:15:023
948035995918,3cyclictest1042896-21ssh00:15:013
94802599585,8cyclictest12-21ksoftirqd/021:41:320
948025995815,11cyclictest12-21ksoftirqd/023:00:400
948025995728,14cyclictest1031341-21ssh23:50:290
948025995727,14cyclictest960073-21ssh21:22:000
948025995717,9cyclictest12-21ksoftirqd/022:35:270
948025995714,7cyclictest12-21ksoftirqd/022:09:350
948032995648,5cyclictest947776-21kworker/2:222:13:462
948032995648,5cyclictest947776-21kworker/2:222:13:462
948032995646,5cyclictest947776-21kworker/2:221:23:142
948027995622,26cyclictest23-21ksoftirqd/123:10:021
948027995622,26cyclictest23-21ksoftirqd/123:10:011
948027995619,10cyclictest23-21ksoftirqd/122:18:201
94802599566,12cyclictest12-21ksoftirqd/023:11:050
94802599564,6cyclictest121ksoftirqd/023:22:310
94802599564,11cyclictest12-21ksoftirqd/021:36:140
948025995622,7cyclictest12-21ksoftirqd/023:42:140
948025995610,4cyclictest131rcu_preempt23:27:230
948032995547,5cyclictest947776-21kworker/2:200:07:372
948032995537,13cyclictest3467602-21systemd-journal22:10:012
948032995531,7cyclictest947776-21kworker/2:223:20:552
948027995538,13cyclictest275-21dbus-daemon23:25:011
94802599558,17cyclictest12-21ksoftirqd/020:35:030
948025995517,34cyclictest12-21ksoftirqd/022:30:030
948025995517,34cyclictest12-21ksoftirqd/022:30:030
948032995446,5cyclictest951349-21kworker/2:122:48:052
948032995446,5cyclictest947776-21kworker/2:223:05:192
948032995446,5cyclictest947776-21kworker/2:223:05:182
948032995446,5cyclictest947776-21kworker/2:219:20:002
948032995446,5cyclictest947776-21kworker/2:200:35:232
948032995445,6cyclictest947776-21kworker/2:222:18:242
948032995445,5cyclictest951349-21kworker/2:121:36:342
948032995445,5cyclictest947776-21kworker/2:200:00:132
948032995444,7cyclictest951349-21kworker/2:122:52:042
948032995444,6cyclictest947776-21kworker/2:222:28:242
948032995444,6cyclictest947776-21kworker/2:222:28:242
94802599546,6cyclictest12-21ksoftirqd/023:40:020
94802599544,7cyclictest12-21ksoftirqd/023:47:040
948035995310,10cyclictest131rcu_preempt00:05:033
948032995345,5cyclictest947776-21kworker/2:222:35:002
948032995344,5cyclictest947776-21kworker/2:200:12:502
948032995344,5cyclictest947776-21kworker/2:200:12:502
94803299533,5cyclictest131rcu_preempt00:00:012
948025995344,5cyclictest16850irq/134-800000023:10:020
948025995344,5cyclictest16850irq/134-800000023:10:020
948025995328,11cyclictest999630-21ssh22:44:450
948025995327,11cyclictest976346-21ssh21:55:540
948025995326,12cyclictest1054796-21ssh00:39:560
948025995322,28cyclictest12-21ksoftirqd/022:13:320
948025995322,28cyclictest12-21ksoftirqd/022:13:310
948032995244,5cyclictest951349-21kworker/2:121:59:412
948032995244,5cyclictest951349-21kworker/2:121:48:252
948032995244,5cyclictest947776-21kworker/2:223:35:562
948032995244,5cyclictest947776-21kworker/2:220:10:022
948032995244,5cyclictest947776-21kworker/2:220:10:012
948032995241,6cyclictest947776-21kworker/2:200:32:342
948032995241,6cyclictest947776-21kworker/2:200:32:342
94802599526,9cyclictest12-21ksoftirqd/021:16:150
948025995231,17cyclictest3467602-21systemd-journal20:17:000
948025995215,3cyclictest996466-21kworker/u8:122:52:430
948035995134,13cyclictest3467602-21systemd-journal23:00:013
948032995143,5cyclictest951349-21kworker/2:122:04:462
948032995143,5cyclictest947776-21kworker/2:223:52:532
948032995143,5cyclictest947776-21kworker/2:200:19:282
948032995142,5cyclictest947776-21kworker/2:223:02:472
948032995142,5cyclictest947776-21kworker/2:222:59:222
948032995141,6cyclictest951349-21kworker/2:121:43:302
948032995136,10cyclictest1-21systemd20:55:012
948032995134,8cyclictest947776-21kworker/2:222:35:092
94802599514,6cyclictest12-21ksoftirqd/022:16:550
948025995130,15cyclictest3467602-21systemd-journal19:55:010
948025995130,15cyclictest3467602-21systemd-journal19:55:000
94802599512,14cyclictest12-21ksoftirqd/019:30:020
948025995118,28cyclictest16850irq/134-800000021:25:370
948035995033,13cyclictest3467602-21systemd-journal23:08:003
948035995033,13cyclictest3467602-21systemd-journal23:08:003
948035995032,13cyclictest1-21systemd23:34:593
948032995042,5cyclictest947776-21kworker/2:223:26:532
948032995042,5cyclictest947776-21kworker/2:221:32:522
948032995042,5cyclictest947776-21kworker/2:221:10:002
948032995041,5cyclictest947776-21kworker/2:221:17:102
948032995041,5cyclictest947776-21kworker/2:221:13:162
948027995043,3cyclictest23-21ksoftirqd/100:20:011
94803599496,6cyclictest131rcu_preempt19:40:023
94803599496,6cyclictest131rcu_preempt19:40:023
94803599495,3cyclictest131rcu_preempt00:00:023
948035994931,14cyclictest1-21systemd19:35:003
948032994942,5cyclictest947776-21kworker/2:223:40:522
948032994941,5cyclictest951349-21kworker/2:121:51:562
948032994941,5cyclictest947776-21kworker/2:222:24:382
948032994941,5cyclictest947776-21kworker/2:222:24:372
948032994941,5cyclictest947776-21kworker/2:220:05:012
948032994941,5cyclictest947776-21kworker/2:200:20:422
948032994940,6cyclictest947776-21kworker/2:223:15:352
948032994940,6cyclictest947776-21kworker/2:223:15:342
948032994940,5cyclictest947776-21kworker/2:223:31:072
948032994940,5cyclictest947776-21kworker/2:219:56:052
948032994940,5cyclictest947776-21kworker/2:200:25:382
948032994939,6cyclictest947776-21kworker/2:222:42:362
948027994929,15cyclictest3467602-21systemd-journal23:40:001
94802599497,9cyclictest12-21ksoftirqd/020:50:000
94802599491,44cyclictest0-21swapper/020:55:020
948025994911,33cyclictest309-21in:imuxsock21:00:000
94803599485,9cyclictest48-21kcompactd022:30:413
948035994832,12cyclictest1-21systemd19:50:013
948035994832,12cyclictest1-21systemd19:44:593
948035994830,13cyclictest3467602-21systemd-journal20:20:003
948032994840,5cyclictest947776-21kworker/2:223:10:502
948032994840,5cyclictest947776-21kworker/2:221:25:362
948032994839,5cyclictest947776-21kworker/2:220:20:012
948032994832,11cyclictest1-21systemd20:25:002
948032994831,12cyclictest1-21systemd20:30:002
94802799483,28cyclictest23-21ksoftirqd/121:35:021
94802599485,36cyclictest12-21ksoftirqd/020:00:000
948025994838,7cyclictest12-21ksoftirqd/019:35:000
94803599477,3cyclictest954191-21diskmemload21:40:013
948032994738,5cyclictest947776-21kworker/2:220:35:012
948027994733,11cyclictest3467602-21systemd-journal22:27:321
948027994733,11cyclictest3467602-21systemd-journal22:27:311
948027994733,10cyclictest994945-21date22:35:021
94803599465,6cyclictest37-21ksoftirqd/321:27:423
94803599463,3cyclictest131rcu_preempt21:20:163
948035994633,10cyclictest1-21systemd23:40:013
948035994615,22cyclictest131rcu_preempt00:36:503
948035994611,6cyclictest131rcu_preempt22:30:033
948035994611,6cyclictest131rcu_preempt22:30:023
948032994638,5cyclictest947776-21kworker/2:221:05:012
948027994636,5cyclictest950527-21cron20:00:001
94802599461,41cyclictest0-21swapper/019:40:010
94802599461,41cyclictest0-21swapper/019:40:000
94803599453,5cyclictest131rcu_preempt21:04:593
948035994528,13cyclictest275-21dbus-daemon23:15:013
948032994538,4cyclictest947776-21kworker/2:219:35:012
94802799453,3cyclictest131rcu_preempt19:40:011
94802799453,3cyclictest131rcu_preempt19:40:001
948027994532,9cyclictest1-21systemd22:45:011
948025994537,5cyclictest16850irq/134-800000020:12:190
948025994526,14cyclictest3467602-21systemd-journal19:50:010
94802599452,38cyclictest954216-21systemd-run21:10:010
94803599447,9cyclictest37-21ksoftirqd/322:53:193
94803599446,5cyclictest131rcu_preempt22:05:023
94803599445,3cyclictest131rcu_preempt21:16:293
94803599443,5cyclictest131rcu_preempt23:55:003
948035994434,8cyclictest37-21ksoftirqd/322:15:013
948035994434,8cyclictest37-21ksoftirqd/322:15:013
948035994434,6cyclictest37-21ksoftirqd/321:00:023
948035994434,5cyclictest954176-21cron21:10:003
948035994430,10cyclictest1-21systemd21:15:003
94803599442,3cyclictest131rcu_preempt23:45:053
94803599441,3cyclictest131rcu_preempt23:30:013
94803599441,3cyclictest131rcu_preempt00:17:563
948032994435,5cyclictest947776-21kworker/2:220:15:012
948032994431,10cyclictest30-21ksoftirqd/220:50:002
948027994434,5cyclictest309-21in:imuxsock00:40:011
94802799441,3cyclictest131rcu_preempt00:10:081
94802799441,3cyclictest131rcu_preempt00:10:081
94803599438,5cyclictest131rcu_preempt23:21:053
948035994331,7cyclictest950526-21cron20:00:003
948035994329,11cyclictest3467602-21systemd-journal23:18:063
948035994329,11cyclictest3467602-21systemd-journal23:18:063
948035994320,5cyclictest37-21ksoftirqd/322:24:453
948035994320,5cyclictest37-21ksoftirqd/322:24:453
948035994318,6cyclictest37-21ksoftirqd/323:43:403
948032994337,3cyclictest947776-21kworker/2:219:35:232
948032994337,3cyclictest947776-21kworker/2:219:35:222
948032994331,8cyclictest30-21ksoftirqd/219:45:002
94802799434,4cyclictest131rcu_preempt00:33:171
94802799434,4cyclictest131rcu_preempt00:33:171
94802799433,3cyclictest131rcu_preempt00:21:431
948027994330,9cyclictest3467602-21systemd-journal19:22:031
948027994329,10cyclictest23-21ksoftirqd/122:05:011
948027994324,14cyclictest275-21dbus-daemon23:30:011
94802799432,3cyclictest131rcu_preempt20:40:011
948027994311,21cyclictest971199-21kworker/u8:321:55:021
94802599431,5cyclictest0-21swapper/020:45:010
94803599423,4cyclictest131rcu_preempt21:52:073
94803599422,4cyclictest131rcu_preempt21:56:423
94803599422,3cyclictest131rcu_preempt20:40:023
94803599421,9cyclictest1040498-21ssh00:10:013
94803599421,5cyclictest131rcu_preempt00:33:233
94803599421,5cyclictest131rcu_preempt00:33:233
94803599421,10cyclictest150-21pr/ttyS222:49:573
948032994235,4cyclictest947776-21kworker/2:219:10:032
948032994234,5cyclictest947776-21kworker/2:219:25:102
948027994231,8cyclictest3467602-21systemd-journal00:27:061
948027994221,16cyclictest3467602-21systemd-journal20:15:001
94802599426,8cyclictest12-21ksoftirqd/020:30:010
94802599425,8cyclictest12-21ksoftirqd/020:05:000
94802599422,9cyclictest12-21ksoftirqd/019:10:020
948025994211,8cyclictest0-21swapper/020:38:580
94803599418,10cyclictest1-21systemd20:45:053
94803599417,6cyclictest131rcu_preempt22:10:023
94803599414,5cyclictest131rcu_preempt19:25:013
94803599413,5cyclictest131rcu_preempt00:25:013
948035994126,12cyclictest948226-21wc19:15:023
948035994126,11cyclictest275-21dbus-daemon20:10:013
948035994126,11cyclictest275-21dbus-daemon20:10:013
94803599411,3cyclictest131rcu_preempt22:43:513
94803599411,3cyclictest131rcu_preempt22:40:003
948032994133,5cyclictest947776-21kworker/2:219:20:132
948027994128,9cyclictest3467602-21systemd-journal23:10:031
948027994126,11cyclictest23-21ksoftirqd/123:53:351
948027994124,12cyclictest275-21dbus-daemon20:50:011
94802599415,8cyclictest12-21ksoftirqd/019:19:590
94803599405,9cyclictest37-21ksoftirqd/321:47:073
94803599405,3cyclictest131rcu_preempt21:35:003
94803599403,4cyclictest131rcu_preempt19:30:033
948035994028,7cyclictest37-21ksoftirqd/320:30:023
94803599402,6cyclictest131rcu_preempt20:35:013
94803599402,3cyclictest131rcu_preempt22:18:253
948032994033,4cyclictest947776-21kworker/2:219:51:222
948032994033,4cyclictest947776-21kworker/2:219:51:222
948032994020,4cyclictest30-21ksoftirqd/221:00:002
948027994026,9cyclictest23-21ksoftirqd/121:50:021
948027994022,14cyclictest1-21systemd20:10:011
948027994022,14cyclictest1-21systemd20:10:001
94802799401,4cyclictest131rcu_preempt23:50:001
94802599401,9cyclictest949868-21cut19:45:020
94803599395,7cyclictest131rcu_preempt19:55:023
94803599395,7cyclictest131rcu_preempt19:55:013
94803599395,6cyclictest131rcu_preempt21:45:013
948035993928,8cyclictest951353-21sed20:15:003
948035993927,10cyclictest37-21ksoftirqd/300:26:013
94803599392,4cyclictest131rcu_preempt20:25:013
94802799396,5cyclictest131rcu_preempt23:15:141
94802799396,5cyclictest131rcu_preempt23:15:141
94802799394,3cyclictest131rcu_preempt22:59:391
948027993931,5cyclictest952110-21sh20:30:001
948027993928,7cyclictest23-21ksoftirqd/121:10:021
948027993924,11cyclictest3467602-21systemd-journal21:05:001
948027993922,12cyclictest309-21in:imuxsock19:15:011
94802799391,5cyclictest0-21swapper/122:05:541
94802799391,4cyclictest131rcu_preempt22:22:191
94802799391,4cyclictest131rcu_preempt22:22:191
94802799391,3cyclictest131rcu_preempt00:05:491
948027993913,5cyclictest131rcu_preempt22:10:201
948027993913,5cyclictest131rcu_preempt22:10:201
948025993925,10cyclictest948178-21sshd19:12:040
94802599391,8cyclictest948794-21latency_hist19:25:020
94802599391,8cyclictest0-21swapper/020:09:240
94802599391,8cyclictest0-21swapper/020:09:240
948027993829,6cyclictest1002231-21ssh22:50:021
948027993828,7cyclictest23-21ksoftirqd/119:35:021
948027993825,9cyclictest23-21ksoftirqd/121:35:311
948027993825,10cyclictest950836-21wc20:05:021
948027993823,11cyclictest1028722-21cut23:45:021
94802799381,7cyclictest0-21swapper/121:40:251
94802799381,3cyclictest131rcu_preempt21:15:001
94803599373,3cyclictest131rcu_preempt23:00:093
94803299373,4cyclictest952596-21modprobe20:37:252
94802799374,4cyclictest131rcu_preempt23:05:001
948027993727,6cyclictest953704-21latency_hist21:00:021
948027993725,9cyclictest948963-21sshd19:25:041
948027993724,9cyclictest275-21dbus-daemon19:45:011
948027993724,10cyclictest284-21systemd-logind21:30:001
94802799371,3cyclictest131rcu_preempt23:34:591
94803599364,3cyclictest131rcu_preempt20:05:023
94803599363,3cyclictest131rcu_preempt19:20:003
94803599361,3cyclictest131rcu_preempt20:45:033
948032993628,5cyclictest947776-21kworker/2:219:50:012
94803299362,5cyclictest131rcu_preempt20:43:042
948032993619,14cyclictest948075-21cat19:10:022
94802799364,3cyclictest131rcu_preempt20:35:021
948027993625,9cyclictest23-21ksoftirqd/122:39:441
94802799358,5cyclictest131rcu_preempt21:55:561
948027993519,11cyclictest309-21in:imuxsock20:20:031
94802799351,3cyclictest131rcu_preempt20:19:591
948035993418,13cyclictest275-21dbus-daemon20:55:013
94803599332,3cyclictest131rcu_preempt19:10:013
948027993322,8cyclictest23-21ksoftirqd/119:52:021
948027993322,8cyclictest23-21ksoftirqd/119:52:021
94802799324,3cyclictest131rcu_preempt20:55:001
94802799315,5cyclictest131rcu_preempt19:20:021
948027993124,4cyclictest948122-21latency_hist19:10:011
948027992918,9cyclictest23-21ksoftirqd/120:45:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional