You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-17 - 06:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Fri Jul 17, 2026 00:45:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9178399933,5cyclictest131rcu_preempt23:45:031
9178399933,5cyclictest131rcu_preempt23:45:021
91783998622,49cyclictest23-21ksoftirqd/123:10:021
91778998681,3cyclictest121ksoftirqd/000:10:020
91778998681,3cyclictest121ksoftirqd/000:10:010
91783997931,10cyclictest309-21in:imuxsock23:47:561
91778997316,48cyclictest12-21ksoftirqd/000:30:010
91792997033,8cyclictest37-21ksoftirqd/322:57:193
9179299695,9cyclictest37-21ksoftirqd/322:50:023
9178799676,9cyclictest30-21ksoftirqd/223:45:012
9178799676,9cyclictest30-21ksoftirqd/223:45:002
91783996750,14cyclictest23-21ksoftirqd/100:30:021
9179299667,11cyclictest37-21ksoftirqd/300:40:013
91792996529,12cyclictest37-21ksoftirqd/321:17:103
91787996420,6cyclictest131rcu_preempt21:15:002
9177899642,7cyclictest275-21dbus-daemon23:50:010
9179299635,9cyclictest37-21ksoftirqd/319:45:023
91792996319,12cyclictest37-21ksoftirqd/300:15:013
91778996340,20cyclictest16850irq/134-800000022:45:010
9179299621,11cyclictest37-21ksoftirqd/322:40:023
91792996116,11cyclictest37-21ksoftirqd/323:35:023
91792996116,11cyclictest37-21ksoftirqd/323:35:013
91778996129,12cyclictest166951-21ssh23:35:000
91778996129,12cyclictest166951-21ssh23:35:000
9177899612,43cyclictest120636-21ssh21:57:170
91778996119,33cyclictest12-21ksoftirqd/022:30:010
91778996116,40cyclictest12-21ksoftirqd/023:23:280
9179299608,10cyclictest37-21ksoftirqd/323:25:003
91792996012,10cyclictest37-21ksoftirqd/323:15:013
9178799608,8cyclictest30-21ksoftirqd/223:40:022
9178799608,11cyclictest30-21ksoftirqd/221:00:012
9178799607,8cyclictest30-21ksoftirqd/222:05:562
9178799605,8cyclictest30-21ksoftirqd/223:10:012
91787996015,11cyclictest30-21ksoftirqd/222:05:012
91787996015,11cyclictest30-21ksoftirqd/222:05:002
91787996011,6cyclictest131rcu_preempt22:57:442
9179299597,9cyclictest37-21ksoftirqd/321:15:023
9179299597,9cyclictest37-21ksoftirqd/320:50:023
91792995914,9cyclictest37-21ksoftirqd/322:11:423
91792995913,5cyclictest131rcu_preempt21:52:353
9178799593,11cyclictest30-21ksoftirqd/223:00:022
91787995917,6cyclictest131rcu_preempt00:20:012
91787995913,11cyclictest30-21ksoftirqd/200:25:002
91778995917,27cyclictest0-21swapper/021:46:150
9179299587,7cyclictest37-21ksoftirqd/320:10:013
9179299585,9cyclictest37-21ksoftirqd/323:55:013
9179299585,7cyclictest37-21ksoftirqd/321:45:023
9179299584,10cyclictest37-21ksoftirqd/322:05:003
9179299584,10cyclictest37-21ksoftirqd/322:05:003
91792995820,13cyclictest37-21ksoftirqd/323:30:003
91792995811,5cyclictest131rcu_preempt21:34:033
9178799583,10cyclictest30-21ksoftirqd/200:10:012
9178799583,10cyclictest30-21ksoftirqd/200:10:012
91787995823,21cyclictest159850-21kworker/2:023:25:022
91778995830,13cyclictest145356-21ssh22:49:100
91778995824,3cyclictest131rcu_preempt21:21:190
91778995819,35cyclictest12-21ksoftirqd/021:38:380
9179299577,8cyclictest37-21ksoftirqd/300:35:003
91792995717,6cyclictest131rcu_preempt23:40:083
91792995717,6cyclictest131rcu_preempt23:40:073
9178799577,11cyclictest30-21ksoftirqd/200:35:012
9178799572,16cyclictest30-21ksoftirqd/200:05:012
91787995710,5cyclictest131rcu_preempt23:34:512
91787995710,5cyclictest131rcu_preempt23:34:502
91778995748,7cyclictest16950irq/224-800000023:15:020
91778995737,15cyclictest16850irq/134-800000022:40:010
91778995728,13cyclictest158905-21ssh23:17:580
91778995728,12cyclictest129444-21ssh22:15:430
9179299566,11cyclictest37-21ksoftirqd/300:15:283
9179299565,10cyclictest37-21ksoftirqd/323:10:003
91792995616,12cyclictest37-21ksoftirqd/300:08:053
91792995616,12cyclictest37-21ksoftirqd/300:08:043
91792995612,6cyclictest131rcu_preempt00:27:093
91787995613,13cyclictest30-21ksoftirqd/223:46:402
91778995628,13cyclictest184402-21ssh00:11:160
91778995625,15cyclictest196727-21ssh00:37:080
9179299558,10cyclictest37-21ksoftirqd/323:55:203
9179299558,10cyclictest37-21ksoftirqd/323:55:193
91792995529,8cyclictest37-21ksoftirqd/321:36:063
9179299551,7cyclictest147936-21munin-run22:55:003
91792995514,9cyclictest37-21ksoftirqd/322:27:043
91792995511,6cyclictest131rcu_preempt21:49:353
91792995510,5cyclictest131rcu_preempt23:37:453
9178799556,8cyclictest30-21ksoftirqd/222:36:262
9178799553,7cyclictest30-21ksoftirqd/219:55:012
9178799552,9cyclictest30-21ksoftirqd/220:35:002
9178799552,8cyclictest95363-21munin-run20:20:002
9178799552,10cyclictest30-21ksoftirqd/221:20:002
91787995515,6cyclictest131rcu_preempt22:45:022
9177899559,3cyclictest12-21ksoftirqd/023:42:030
9177899559,3cyclictest12-21ksoftirqd/023:42:020
91778995528,12cyclictest181093-21ssh00:04:400
91778995526,13cyclictest163619-21ssh23:27:500
9177899552,37cyclictest174882-21ssh23:51:190
9179299546,11cyclictest37-21ksoftirqd/322:00:003
9179299546,10cyclictest37-21ksoftirqd/320:20:003
91792995425,10cyclictest37-21ksoftirqd/319:20:013
9178799546,11cyclictest30-21ksoftirqd/220:25:022
9178799543,9cyclictest30-21ksoftirqd/222:00:012
9178799543,13cyclictest30-21ksoftirqd/221:30:002
91787995413,5cyclictest131rcu_preempt22:30:102
9178399541,37cyclictest1-21systemd22:05:021
9178399541,37cyclictest1-21systemd22:05:011
9177899546,40cyclictest12-21ksoftirqd/023:39:270
91778995422,16cyclictest137740-21ssh22:33:220
91778995419,31cyclictest12-21ksoftirqd/000:20:120
91778995413,5cyclictest131rcu_preempt23:06:180
9179299537,8cyclictest37-21ksoftirqd/322:43:573
9179299536,6cyclictest37-21ksoftirqd/323:50:013
9179299535,8cyclictest37-21ksoftirqd/322:24:273
9179299534,12cyclictest37-21ksoftirqd/322:08:373
9179299534,10cyclictest37-21ksoftirqd/323:01:553
9178799537,10cyclictest30-21ksoftirqd/200:28:392
9178799534,9cyclictest30-21ksoftirqd/220:40:002
91787995344,5cyclictest102788-21kworker/2:221:54:592
91787995336,12cyclictest275-21dbus-daemon00:15:002
9178799532,18cyclictest30-21ksoftirqd/220:05:012
9178799531,9cyclictest30-21ksoftirqd/200:00:012
9178799531,9cyclictest30-21ksoftirqd/200:00:002
91787995318,26cyclictest91773-21cyclictest21:35:012
91787995311,5cyclictest131rcu_preempt22:47:142
91778995334,14cyclictest1-21systemd19:15:000
91778995326,12cyclictest101202-21ssh21:16:300
91778995325,12cyclictest125118-21ssh22:06:390
91778995323,14cyclictest119112-21ssh21:54:170
9177899531,3cyclictest0-21swapper/021:00:010
9179299527,8cyclictest37-21ksoftirqd/321:28:323
9179299527,8cyclictest37-21ksoftirqd/321:25:013
9179299526,8cyclictest37-21ksoftirqd/321:10:023
9179299524,9cyclictest37-21ksoftirqd/322:31:083
91792995214,6cyclictest131rcu_preempt00:04:113
9178799527,9cyclictest30-21ksoftirqd/222:28:202
9178799525,9cyclictest30-21ksoftirqd/222:16:302
9178799525,7cyclictest30-21ksoftirqd/220:15:022
9178799524,6cyclictest131rcu_preempt21:45:002
9178799524,10cyclictest30-21ksoftirqd/223:11:092
91783995234,14cyclictest3467602-21systemd-journal20:55:011
91778995222,26cyclictest364-21snmpd23:57:580
91778995222,26cyclictest364-21snmpd23:57:570
9177899521,36cyclictest105715-21ssh21:25:540
9179299517,11cyclictest37-21ksoftirqd/322:19:593
9179299516,8cyclictest37-21ksoftirqd/320:35:023
9179299512,9cyclictest37-21ksoftirqd/300:20:503
9178799519,6cyclictest131rcu_preempt22:54:592
9178799519,5cyclictest131rcu_preempt21:36:052
9178799518,6cyclictest131rcu_preempt19:40:012
9178799516,8cyclictest30-21ksoftirqd/200:35:522
9178799515,5cyclictest131rcu_preempt23:16:212
91787995144,4cyclictest95668-21kworker/2:322:24:472
91787995115,6cyclictest131rcu_preempt23:52:462
91787995111,6cyclictest131rcu_preempt20:49:592
91787995111,5cyclictest131rcu_preempt21:22:112
91783995136,11cyclictest275-21dbus-daemon23:55:021
91783995132,15cyclictest169347-21systemctl23:39:591
91778995127,20cyclictest99898-21rm21:13:580
91792995012,6cyclictest131rcu_preempt19:50:013
91792995010,10cyclictest37-21ksoftirqd/321:00:013
9178799508,5cyclictest131rcu_preempt20:10:012
9178799505,9cyclictest30-21ksoftirqd/219:25:002
91787995042,5cyclictest97463-21kworker/2:121:07:282
91787995042,5cyclictest95668-21kworker/2:321:47:412
91787995042,5cyclictest82136-21kworker/2:119:33:122
91787995011,5cyclictest131rcu_preempt19:10:012
91778995033,14cyclictest19350irq/34-mmc122:55:020
91778995025,18cyclictest275-21dbus-daemon20:00:010
91778995025,10cyclictest195592-21ssh00:34:530
9179299498,7cyclictest37-21ksoftirqd/321:05:023
9179299498,7cyclictest37-21ksoftirqd/320:39:593
9179299496,8cyclictest37-21ksoftirqd/320:25:023
9179299496,11cyclictest37-21ksoftirqd/323:19:053
9179299495,8cyclictest37-21ksoftirqd/320:55:003
91792994911,6cyclictest131rcu_preempt20:05:013
9178799495,7cyclictest30-21ksoftirqd/220:45:012
91787994941,5cyclictest82136-21kworker/2:119:11:262
91787994941,5cyclictest102788-21kworker/2:222:11:462
91787994918,6cyclictest131rcu_preempt23:29:182
91787994913,6cyclictest131rcu_preempt20:00:022
91778994925,9cyclictest122605-21ssh22:01:140
91778994925,9cyclictest122605-21ssh22:01:130
91778994921,13cyclictest113311-21ssh21:41:500
91778994921,11cyclictest128583-21ssh22:14:110
9179299487,7cyclictest37-21ksoftirqd/319:15:003
9179299484,7cyclictest37-21ksoftirqd/319:35:023
9179299483,7cyclictest37-21ksoftirqd/320:30:013
9179299481,7cyclictest96669-21munin-plugin-st20:45:003
91787994840,5cyclictest91749-21kworker/2:219:41:032
91787994840,5cyclictest82136-21kworker/2:119:17:342
91787994839,6cyclictest94872-21kworker/2:020:28:072
9177899489,36cyclictest131rcu_preempt22:58:560
9177899481,17cyclictest151699-21ssh23:02:450
9179299477,11cyclictest37-21ksoftirqd/319:52:173
9179299474,9cyclictest37-21ksoftirqd/319:30:003
91792994710,5cyclictest131rcu_preempt20:15:013
9178799475,7cyclictest30-21ksoftirqd/219:30:002
91787994739,5cyclictest97463-21kworker/2:121:03:162
91787994739,5cyclictest95668-21kworker/2:320:50:512
91787994739,5cyclictest82136-21kworker/2:119:47:322
91783994732,12cyclictest23-21ksoftirqd/100:00:121
91783994731,11cyclictest1-21systemd00:15:011
91783994729,14cyclictest3467602-21systemd-journal21:31:221
91778994738,6cyclictest91785-21latency_hist19:10:020
91778994738,5cyclictest16950irq/224-800000020:50:000
91778994737,8cyclictest0-21swapper/019:20:000
9177899472,40cyclictest133649-21ssh22:25:010
9177899472,3cyclictest131rcu_preempt21:30:380
9179299465,10cyclictest37-21ksoftirqd/320:00:003
91783994627,13cyclictest3467602-21systemd-journal00:05:031
91783994627,13cyclictest3467602-21systemd-journal00:05:021
91783994615,7cyclictest131rcu_preempt19:40:001
91778994633,10cyclictest12-21ksoftirqd/019:40:020
9179299458,5cyclictest131rcu_preempt19:10:013
91783994533,8cyclictest23-21ksoftirqd/121:50:011
9178399452,3cyclictest131rcu_preempt23:03:411
9179299444,6cyclictest131rcu_preempt19:40:023
91783994432,7cyclictest0-21swapper/119:10:011
9177899444,6cyclictest131rcu_preempt19:35:010
91778994430,11cyclictest19350irq/34-mmc120:35:020
91778994426,13cyclictest275-21dbus-daemon19:25:000
91778994421,18cyclictest12-21ksoftirqd/000:15:040
91792994334,7cyclictest361rcuc/319:20:473
9178399439,6cyclictest131rcu_preempt22:25:021
9178399438,6cyclictest131rcu_preempt19:20:001
9178399434,8cyclictest131rcu_preempt19:35:001
9178399433,3cyclictest131rcu_preempt22:41:061
91783994331,8cyclictest23-21ksoftirqd/121:16:211
91783994328,11cyclictest179079-21sed00:00:031
91783994328,11cyclictest179079-21sed00:00:021
91783994327,12cyclictest275-21dbus-daemon19:25:001
9178399432,4cyclictest131rcu_preempt00:15:171
91778994333,6cyclictest95356-21sh20:20:010
91778994327,11cyclictest309-21in:imuxsock19:50:020
9177899431,36cyclictest311-21rs:main0
91783994229,9cyclictest23-21ksoftirqd/121:40:471
91783994229,10cyclictest23-21ksoftirqd/123:10:031
9178399422,3cyclictest131rcu_preempt22:57:161
9178399421,5cyclictest131rcu_preempt22:35:001
91778994235,4cyclictest16950irq/224-800000021:10:000
9177899422,6cyclictest96671-21munin-run20:45:010
91778994226,12cyclictest0-21swapper/019:45:010
9178399413,4cyclictest131rcu_preempt23:34:051
9178399413,4cyclictest131rcu_preempt23:34:051
91783994128,10cyclictest23-21ksoftirqd/121:27:111
91783994126,11cyclictest23-21ksoftirqd/119:50:011
91783994125,12cyclictest3467602-21systemd-journal20:25:021
91783994124,12cyclictest275-21dbus-daemon21:55:001
9178399412,17cyclictest0-21swapper/100:25:161
91778994133,4cyclictest16950irq/224-800000020:40:000
91783994028,10cyclictest23-21ksoftirqd/121:56:131
91783994025,12cyclictest23-21ksoftirqd/122:52:221
91783994025,10cyclictest23-21ksoftirqd/120:50:001
91783994022,14cyclictest1-21systemd22:50:011
91783994017,4cyclictest23-21ksoftirqd/120:00:031
9178399401,3cyclictest131rcu_preempt22:26:311
91778994029,8cyclictest94687-21cat20:05:020
91783993931,5cyclictest96667-21latency_hist20:45:001
91783993927,9cyclictest23-21ksoftirqd/121:14:171
91783993927,10cyclictest23-21ksoftirqd/122:19:591
91783993926,10cyclictest93755-21sed19:45:031
91783993925,11cyclictest275-21dbus-daemon00:40:001
9177899394,7cyclictest3467718-21lldpd19:25:330
9177899392,8cyclictest97302-21latency_hist20:55:010
91778993925,10cyclictest93786-21sshd19:48:220
9178399384,3cyclictest131rcu_preempt23:27:251
91783993830,5cyclictest159812-21latency_hist23:20:001
91783993828,8cyclictest23-21ksoftirqd/120:10:021
91783993825,10cyclictest23-21ksoftirqd/122:10:331
91783993825,10cyclictest23-21ksoftirqd/121:04:531
91783993824,11cyclictest23-21ksoftirqd/122:07:421
91783993823,10cyclictest275-21dbus-daemon20:40:011
9178399381,3cyclictest131rcu_preempt23:24:371
9177899381,33cyclictest97679-21sshd21:02:210
9178399377,7cyclictest23-21ksoftirqd/119:13:221
9178399377,3cyclictest131rcu_preempt00:24:491
91783993726,8cyclictest23-21ksoftirqd/121:25:021
9178399372,4cyclictest131rcu_preempt20:15:001
91783993719,14cyclictest3467602-21systemd-journal20:20:011
91778993726,8cyclictest0-21swapper/020:10:010
91778993725,10cyclictest12-21ksoftirqd/020:15:010
91783993629,4cyclictest95883-21latency_hist20:30:001
9178399362,4cyclictest131rcu_preempt21:38:221
9178399361,12cyclictest139770-21sh22:37:351
9177899367,4cyclictest131rcu_preempt20:25:010
9178399353,4cyclictest131rcu_preempt19:30:011
91783993524,8cyclictest23-21ksoftirqd/119:52:411
9178399352,3cyclictest131rcu_preempt21:10:011
9178399351,7cyclictest364-21snmpd20:03:481
9178399351,4cyclictest131rcu_preempt21:00:001
91783993225,5cyclictest96135-21latency_hist20:35:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional