You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-19 - 03:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Sun Jul 19, 2026 00:45:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
747521999184,3cyclictest0-21swapper/000:30:020
747521997327,23cyclictest833794-21ssh00:00:050
747521997313,44cyclictest833495-40mandb00:00:030
747521997259,9cyclictest796201-21sed22:40:020
747533997149,19cyclictest30-21ksoftirqd/200:25:022
747525997115,9cyclictest23-21ksoftirqd/121:55:011
747535996734,9cyclictest37-21ksoftirqd/300:00:513
747533996723,9cyclictest833551-21rm00:00:012
747533996623,4cyclictest749502-21latency_hist19:45:022
747533996622,8cyclictest30-21ksoftirqd/200:06:262
74752199669,39cyclictest16950irq/224-800000023:10:030
747533996525,5cyclictest774647-21sh21:54:452
747521996530,3cyclictest121ksoftirqd/021:20:010
747535996416,5cyclictest131rcu_preempt22:40:023
747521996436,6cyclictest12-21ksoftirqd/021:10:030
747533996327,5cyclictest777141-21latency_hist22:00:022
747533996322,6cyclictest849988-21munin-plugin-st00:35:002
747533996322,6cyclictest849988-21munin-plugin-st00:35:002
747533996126,4cyclictest819218-21sh23:29:202
747533996124,5cyclictest835310-21sh00:03:422
747525996113,6cyclictest131rcu_preempt21:14:061
747521996150,8cyclictest12-21ksoftirqd/021:55:270
74752599605,6cyclictest131rcu_preempt00:00:001
74752599605,14cyclictest23-21ksoftirqd/122:40:001
747525996013,5cyclictest131rcu_preempt20:25:001
747521996024,32cyclictest12-21ksoftirqd/023:33:090
747521996013,43cyclictest12-21ksoftirqd/023:39:390
747533995925,6cyclictest30-21ksoftirqd/200:27:062
74752199593,6cyclictest131rcu_preempt23:55:010
74752199593,5cyclictest131rcu_preempt00:36:560
747521995929,14cyclictest825293-21ssh23:42:000
747521995916,34cyclictest131rcu_preempt00:25:010
747525995819,9cyclictest23-21ksoftirqd/100:35:011
747525995819,9cyclictest23-21ksoftirqd/100:35:001
747525995815,5cyclictest131rcu_preempt00:06:051
747525995812,6cyclictest131rcu_preempt20:45:021
747521995830,12cyclictest797938-21ssh22:44:010
747521995829,14cyclictest800676-21ssh22:49:580
747521995829,13cyclictest783959-21ssh22:14:210
747521995822,7cyclictest12-21ksoftirqd/000:08:210
74753599573,7cyclictest37-21ksoftirqd/300:00:033
747535995724,10cyclictest37-21ksoftirqd/323:45:333
747533995725,10cyclictest3467602-21systemd-journal22:24:292
747533995719,11cyclictest30-21ksoftirqd/222:27:592
747533995716,13cyclictest30-21ksoftirqd/220:35:002
74752599578,6cyclictest131rcu_preempt00:15:021
74752599577,9cyclictest23-21ksoftirqd/121:10:001
747525995714,6cyclictest131rcu_preempt21:40:041
747525995714,6cyclictest131rcu_preempt21:40:041
747525995711,6cyclictest131rcu_preempt23:15:591
747525995710,7cyclictest131rcu_preempt22:10:011
747521995742,12cyclictest12-21ksoftirqd/000:19:240
747521995726,15cyclictest811536-21ssh23:12:450
747521995726,15cyclictest811536-21ssh23:12:450
747533995629,8cyclictest777195-21kworker/2:022:03:212
747533995626,5cyclictest822369-21sh23:35:352
747533995625,8cyclictest30-21ksoftirqd/221:05:022
747533995625,4cyclictest810063-21sh23:09:492
74752599567,11cyclictest23-21ksoftirqd/123:28:591
747525995624,11cyclictest23-21ksoftirqd/123:50:001
747525995613,5cyclictest131rcu_preempt23:30:181
747521995652,2cyclictest0-21swapper/000:10:040
747521995627,13cyclictest781597-21ssh22:09:230
74752199562,14cyclictest788961-21ssh22:25:000
747521995620,32cyclictest12-21ksoftirqd/021:50:000
74753599558,6cyclictest131rcu_preempt23:31:353
747535995545,6cyclictest37-21ksoftirqd/300:38:213
747535995510,11cyclictest37-21ksoftirqd/321:50:003
74752599558,6cyclictest131rcu_preempt22:45:021
747525995514,6cyclictest131rcu_preempt23:20:531
747525995514,6cyclictest131rcu_preempt23:20:531
747525995514,5cyclictest131rcu_preempt22:55:021
747525995513,5cyclictest131rcu_preempt21:35:011
747525995513,5cyclictest131rcu_preempt21:29:411
747521995527,13cyclictest802031-21ssh22:52:400
747521995519,33cyclictest12-21ksoftirqd/023:18:520
747521995516,28cyclictest0-21swapper/021:45:030
747521995516,28cyclictest0-21swapper/021:45:020
747535995427,17cyclictest37-21ksoftirqd/300:07:043
747533995443,7cyclictest748788-21kworker/2:321:49:272
747533995427,4cyclictest842516-21sh00:19:102
747525995416,5cyclictest131rcu_preempt23:37:121
747525995415,10cyclictest23-21ksoftirqd/100:36:191
747525995413,9cyclictest131rcu_preempt23:09:431
747525995412,6cyclictest131rcu_preempt21:20:001
74752199547,28cyclictest16950irq/224-800000023:00:030
747535995319,19cyclictest0-21swapper/323:42:153
747535995313,16cyclictest131rcu_preempt21:13:463
747533995328,6cyclictest30-21ksoftirqd/222:51:542
747533995323,7cyclictest30-21ksoftirqd/221:40:022
747525995317,6cyclictest131rcu_preempt22:25:501
747525995315,5cyclictest131rcu_preempt21:21:301
747525995312,6cyclictest131rcu_preempt23:11:061
747525995312,6cyclictest131rcu_preempt23:11:061
747525995310,5cyclictest131rcu_preempt22:13:461
74752199531,41cyclictest0-21swapper/021:39:420
74753599527,6cyclictest131rcu_preempt00:15:023
747535995241,8cyclictest37-21ksoftirqd/322:10:013
747535995220,19cyclictest37-21ksoftirqd/323:09:103
747535995210,6cyclictest131rcu_preempt22:50:023
747533995242,6cyclictest748788-21kworker/2:323:30:212
747533995242,6cyclictest748788-21kworker/2:300:40:002
747533995241,6cyclictest748788-21kworker/2:323:02:312
74752599526,9cyclictest23-21ksoftirqd/122:16:311
747525995214,6cyclictest131rcu_preempt22:59:461
747525995214,6cyclictest131rcu_preempt00:21:561
747525995214,5cyclictest131rcu_preempt00:00:101
747525995213,9cyclictest131rcu_preempt20:55:291
747521995222,26cyclictest793249-21ssh22:33:550
74752199521,46cyclictest753183-21munin-run21:00:000
74753599513,9cyclictest37-21ksoftirqd/300:25:003
747535995124,10cyclictest37-21ksoftirqd/322:20:243
747535995123,7cyclictest37-21ksoftirqd/321:25:063
747535995122,9cyclictest37-21ksoftirqd/322:13:573
747535995113,6cyclictest131rcu_preempt23:18:483
747535995111,6cyclictest131rcu_preempt21:20:023
747535995111,6cyclictest131rcu_preempt21:10:003
747533995143,5cyclictest748788-21kworker/2:322:38:362
747533995142,6cyclictest741248-21kworker/2:019:25:032
747533995131,14cyclictest275-21dbus-daemon23:20:002
74752599519,6cyclictest131rcu_preempt23:52:171
74752599519,5cyclictest131rcu_preempt21:55:261
74752599515,12cyclictest23-21ksoftirqd/122:02:591
74752599514,6cyclictest131rcu_preempt21:40:031
747525995116,7cyclictest131rcu_preempt22:32:111
747525995116,5cyclictest131rcu_preempt00:16:191
747525995112,6cyclictest131rcu_preempt23:44:361
747525995111,6cyclictest131rcu_preempt20:30:001
747525995111,6cyclictest131rcu_preempt19:15:011
747521995110,5cyclictest131rcu_preempt00:34:050
747521995110,5cyclictest131rcu_preempt00:34:050
74753599509,5cyclictest131rcu_preempt22:00:023
74753599506,7cyclictest37-21ksoftirqd/322:53:193
74753599506,6cyclictest37-21ksoftirqd/300:28:373
747535995021,10cyclictest37-21ksoftirqd/300:19:203
747535995011,6cyclictest131rcu_preempt22:27:543
747535995010,6cyclictest131rcu_preempt20:30:003
747535995010,5cyclictest131rcu_preempt20:25:023
747533995042,5cyclictest781851-21kworker/2:100:12:182
747533995042,5cyclictest748788-21kworker/2:321:32:122
747533995041,5cyclictest781851-21kworker/2:122:31:042
747533995034,12cyclictest30-21ksoftirqd/220:40:022
74752599508,9cyclictest23-21ksoftirqd/100:25:061
74752599508,5cyclictest131rcu_preempt21:48:291
74752599502,10cyclictest23-21ksoftirqd/122:47:391
74752199505,4cyclictest12-21ksoftirqd/022:15:110
747521995043,4cyclictest773844-21sh21:52:540
747521995022,13cyclictest805734-21ssh23:00:060
74753599497,6cyclictest37-21ksoftirqd/323:01:213
74753599492,9cyclictest37-21ksoftirqd/323:14:473
74753599492,9cyclictest37-21ksoftirqd/323:14:473
747535994914,5cyclictest131rcu_preempt21:00:003
747535994912,6cyclictest131rcu_preempt00:33:253
747535994912,6cyclictest131rcu_preempt00:33:243
747535994911,5cyclictest131rcu_preempt23:00:003
747533994941,5cyclictest748788-21kworker/2:322:05:452
747533994941,4cyclictest749580-21kworker/2:221:19:142
747533994940,6cyclictest748788-21kworker/2:321:26:362
747533994939,6cyclictest748788-21kworker/2:323:40:422
747533994933,12cyclictest3467602-21systemd-journal19:50:032
747533994933,11cyclictest3467602-21systemd-journal20:10:002
747525994914,5cyclictest131rcu_preempt23:02:311
747525994910,6cyclictest131rcu_preempt22:21:531
747521994921,15cyclictest12-21ksoftirqd/022:28:330
747521994921,12cyclictest763921-21ssh21:31:180
747521994914,4cyclictest753135-21kworker/u8:121:25:020
74752199491,3cyclictest131rcu_preempt23:29:020
74753599489,6cyclictest131rcu_preempt22:20:023
74753599486,5cyclictest131rcu_preempt22:35:023
74753599481,4cyclictest131rcu_preempt23:37:223
747535994813,6cyclictest131rcu_preempt23:22:003
747535994813,6cyclictest131rcu_preempt23:22:003
747533994840,5cyclictest781851-21kworker/2:123:22:282
747533994840,5cyclictest781851-21kworker/2:123:22:282
747533994840,5cyclictest777195-21kworker/2:022:10:092
747533994840,5cyclictest748788-21kworker/2:321:40:512
747533994840,5cyclictest748788-21kworker/2:321:40:512
747533994840,5cyclictest748788-21kworker/2:321:12:122
747533994839,6cyclictest781851-21kworker/2:122:16:132
747533994839,5cyclictest781851-21kworker/2:122:45:112
74753399482,9cyclictest131rcu_preempt22:45:022
74753399481,5cyclictest131rcu_preempt23:10:302
74753399481,5cyclictest131rcu_preempt23:10:292
74752599487,8cyclictest23-21ksoftirqd/119:50:001
74752599487,8cyclictest23-21ksoftirqd/119:50:001
747521994820,13cyclictest759734-21ssh21:22:350
74753599479,6cyclictest131rcu_preempt21:30:213
74753599476,10cyclictest37-21ksoftirqd/320:10:023
747535994732,11cyclictest3467602-21systemd-journal22:43:293
74753599472,5cyclictest131rcu_preempt20:20:013
74753599471,5cyclictest131rcu_preempt21:54:033
747535994712,5cyclictest131rcu_preempt23:51:273
74753399471,33cyclictest746704-21kworker/u8:319:15:002
74752599479,5cyclictest131rcu_preempt19:10:031
74752599479,5cyclictest131rcu_preempt19:10:021
74752599478,6cyclictest131rcu_preempt19:40:021
74752599476,11cyclictest131rcu_preempt19:33:301
747525994715,5cyclictest131rcu_preempt20:50:011
747525994715,5cyclictest131rcu_preempt20:50:011
747521994738,5cyclictest16950irq/224-800000019:40:010
747521994738,5cyclictest131rcu_preempt19:30:010
74752199473,5cyclictest131rcu_preempt23:50:000
74753599467,5cyclictest131rcu_preempt21:36:573
74753599466,8cyclictest37-21ksoftirqd/319:10:043
74753599466,8cyclictest37-21ksoftirqd/319:10:043
747535994637,6cyclictest37-21ksoftirqd/320:55:003
74753599462,8cyclictest37-21ksoftirqd/319:20:003
74753599462,4cyclictest131rcu_preempt20:05:013
74753599462,4cyclictest131rcu_preempt20:05:003
747533994640,3cyclictest748788-21kworker/2:321:05:292
747533994638,5cyclictest748788-21kworker/2:323:45:292
747533994638,5cyclictest748788-21kworker/2:322:57:132
747533994638,5cyclictest739623-21kworker/2:119:17:002
74752599465,8cyclictest23-21ksoftirqd/120:35:001
74752599464,8cyclictest23-21ksoftirqd/120:40:001
74752599463,5cyclictest131rcu_preempt19:30:021
747525994610,5cyclictest131rcu_preempt19:55:021
74752199461,4cyclictest131rcu_preempt19:32:300
74753599459,6cyclictest131rcu_preempt22:00:223
74753599458,9cyclictest37-21ksoftirqd/321:40:463
74753599458,9cyclictest37-21ksoftirqd/321:40:463
74753599458,6cyclictest131rcu_preempt19:15:013
747535994528,12cyclictest275-21dbus-daemon20:00:013
747535994514,18cyclictest37-21ksoftirqd/319:41:333
74752599455,6cyclictest131rcu_preempt20:55:001
74752599453,12cyclictest23-21ksoftirqd/120:00:011
74752599452,6cyclictest23-21ksoftirqd/119:45:011
747525994511,5cyclictest131rcu_preempt20:10:011
74752199457,20cyclictest131rcu_preempt23:20:410
74752199457,20cyclictest131rcu_preempt23:20:410
747521994528,12cyclictest753715-21cut21:10:000
74753599449,5cyclictest131rcu_preempt21:23:273
747535994429,10cyclictest275-21dbus-daemon23:30:003
74753399442,13cyclictest131rcu_preempt20:55:012
74752599446,6cyclictest131rcu_preempt20:05:021
74752599446,6cyclictest131rcu_preempt20:05:021
747525994432,8cyclictest23-21ksoftirqd/120:20:001
747525994412,6cyclictest131rcu_preempt19:20:001
747521994420,11cyclictest777586-21ssh22:00:270
74753599436,6cyclictest131rcu_preempt19:35:013
74753599433,6cyclictest37-21ksoftirqd/320:40:023
74752599435,6cyclictest131rcu_preempt21:03:581
747525994321,9cyclictest23-21ksoftirqd/120:14:291
74752199435,5cyclictest131rcu_preempt19:50:010
74752199435,5cyclictest131rcu_preempt19:50:010
74752199431,5cyclictest131rcu_preempt21:05:010
74753599425,9cyclictest37-21ksoftirqd/320:35:003
747535994212,4cyclictest131rcu_preempt19:40:023
747533994234,5cyclictest741248-21kworker/2:019:40:002
747521994224,14cyclictest752385-21sshd20:41:290
74753599414,8cyclictest37-21ksoftirqd/320:12:293
74753599414,7cyclictest37-21ksoftirqd/320:50:023
74753599414,7cyclictest37-21ksoftirqd/320:50:023
747535994129,8cyclictest3467602-21systemd-journal19:30:003
747535994115,17cyclictest37-21ksoftirqd/319:23:043
747535994110,5cyclictest131rcu_preempt20:44:293
747533994133,6cyclictest781851-21kworker/2:123:51:452
74752599415,7cyclictest23-21ksoftirqd/119:25:011
747521994134,4cyclictest16950irq/224-800000019:10:290
747521994131,7cyclictest12-21ksoftirqd/020:50:020
747521994131,7cyclictest12-21ksoftirqd/020:50:020
747521994126,10cyclictest751084-21cron20:20:000
747535994014,15cyclictest37-21ksoftirqd/319:50:043
747533994022,13cyclictest750306-21sshd20:02:252
747533994022,13cyclictest750306-21sshd20:02:252
747521994032,5cyclictest12-21ksoftirqd/020:36:560
74752199401,6cyclictest131rcu_preempt19:20:000
747533993927,7cyclictest761006-21date21:25:022
747533993922,11cyclictest275-21dbus-daemon19:50:022
747533993922,11cyclictest275-21dbus-daemon19:50:012
747521993924,11cyclictest3467602-21systemd-journal19:57:290
747521993922,4cyclictest12-21ksoftirqd/020:05:030
747521993916,19cyclictest12-21ksoftirqd/019:25:010
747535993824,10cyclictest275-21dbus-daemon21:05:003
747535993823,11cyclictest37-21ksoftirqd/319:50:023
747535993823,11cyclictest37-21ksoftirqd/319:50:023
747533993828,6cyclictest748788-21kworker/2:320:45:012
747533993826,9cyclictest309-21in:imuxsock19:30:022
74752199382,5cyclictest3467602-21systemd-journal19:51:450
747521993825,10cyclictest12-21ksoftirqd/020:10:000
74753399376,6cyclictest131rcu_preempt20:30:002
74753399374,9cyclictest30-21ksoftirqd/219:30:002
74752199374,25cyclictest747522-21latency_hist19:10:020
74752199374,25cyclictest747522-21latency_hist19:10:020
74752199373,5cyclictest131rcu_preempt20:05:020
747521993729,5cyclictest309-21in:imuxsock20:20:020
74752199371,5cyclictest131rcu_preempt20:55:000
74752199371,5cyclictest131rcu_preempt20:30:020
747533993626,8cyclictest30-21ksoftirqd/220:10:022
747533993625,8cyclictest30-21ksoftirqd/220:20:292
747533993618,14cyclictest751098-21cron20:20:002
747521993627,3cyclictest12-21ksoftirqd/019:45:010
747533993525,6cyclictest750040-21cron20:00:002
74753399351,9cyclictest752679-21unin-run20:50:012
74753399351,9cyclictest752679-21unin-run20:50:012
74752199354,4cyclictest131rcu_preempt20:15:010
747521993518,14cyclictest751973-21sed20:35:020
74753399331,3cyclictest131rcu_preempt21:00:002
74753399302,5cyclictest131rcu_preempt19:10:022
74753399302,5cyclictest131rcu_preempt19:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional