You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-21 - 16:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Fri Aug 21, 2026 12:45:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3027526999526,15cyclictest3115569-21ssh12:03:080
3027526998928,12cyclictest3053882-21ssh09:52:490
3027526998725,8cyclictest3100108-21ssh11:30:130
302752699851,39cyclictest3125516-21ssh12:24:090
3027526998325,13cyclictest3056722-21ssh09:58:550
3027526998228,13cyclictest3068316-21ssh10:23:240
3027526998223,7cyclictest3110619-21ssh11:52:460
3027526998128,15cyclictest3104930-21ssh11:40:300
3027526998121,15cyclictest3066479-21ssh10:19:440
3027526998120,15cyclictest3122520-21ssh12:17:460
3027526998027,8cyclictest3086322-21ssh11:01:110
3027526998027,35cyclictest16850irq/134-800000009:50:020
3027526998025,9cyclictest3037289-21ssh09:17:240
3027526997975,2cyclictest0-21swapper/011:20:020
3027526997824,13cyclictest3083192-21ssh10:55:010
3027526997824,13cyclictest3083192-21ssh10:55:000
3027526997824,11cyclictest3084076-21ssh10:56:350
302752699781,29cyclictest48-21kcompactd011:45:310
3027526997726,11cyclictest3075423-21ssh10:38:270
3027526997725,7cyclictest3116687-21ssh12:05:190
3027526997724,12cyclictest3072798-21ssh10:32:450
3027526997546,26cyclictest0-21swapper/011:55:020
302752699752,9cyclictest131rcu_preempt07:35:020
3027526997528,14cyclictest3060526-21ssh10:06:400
3027526997528,14cyclictest3060526-21ssh10:06:400
3027526997526,12cyclictest3062538-21ssh10:10:530
302752699743,63cyclictest309-21in:imuxsock09:15:020
3027526997431,27cyclictest3043746-21ssh09:31:040
302752699742,7cyclictest3467602-21systemd-journal12:11:060
3027526997424,11cyclictest3038909-21ssh09:20:360
3027526997414,13cyclictest3078466-21ssh10:44:560
3027526997225,15cyclictest3093138-21ssh11:15:440
3027526997223,11cyclictest3089226-21ssh11:07:290
3027526997167,2cyclictest0-21swapper/007:40:020
302752699711,67cyclictest0-21swapper/008:30:020
3027526997065,2cyclictest0-21swapper/011:25:030
3027526997028,8cyclictest3132300-21ssh12:38:430
3027526997028,12cyclictest12-21ksoftirqd/010:04:250
3027526996929,36cyclictest16950irq/224-800000011:39:590
3027526996830,34cyclictest16950irq/224-800000012:35:010
3027529996716,12cyclictest23-21ksoftirqd/112:00:011
302752699672,6cyclictest1-21systemd07:20:000
3027526996726,18cyclictest1-21systemd09:45:010
3027534996542,7cyclictest3080878-21kworker/2:111:41:482
3027526996526,12cyclictest3042710-21ssh09:29:090
3027526996526,12cyclictest3042710-21ssh09:29:090
3027526996429,30cyclictest16950irq/224-800000009:10:000
3027526996420,11cyclictest3126628-21ssh12:26:200
3027526996419,10cyclictest3027697-21cron07:15:000
302752999635,9cyclictest23-21ksoftirqd/111:45:021
3027534996235,8cyclictest3044874-21kworker/2:010:46:122
3027529996223,13cyclictest23-21ksoftirqd/112:24:471
3027529996221,11cyclictest0-21swapper/110:35:191
3027534996145,7cyclictest3080878-21kworker/2:111:25:552
3027529996113,8cyclictest23-21ksoftirqd/112:16:071
3027526996122,7cyclictest3071383-21rm10:29:570
3027529996041,15cyclictest1-21systemd10:20:021
3027534995924,26cyclictest0-21swapper/208:15:032
3027534995924,26cyclictest0-21swapper/208:15:022
3027529995950,6cyclictest3033690-21diskmemload12:15:021
3027529995919,8cyclictest23-21ksoftirqd/107:25:021
302752999589,8cyclictest23-21ksoftirqd/109:05:011
3027529995817,9cyclictest23-21ksoftirqd/111:15:001
3027529995817,9cyclictest23-21ksoftirqd/111:06:441
3027529995811,11cyclictest23-21ksoftirqd/110:45:021
3027526995830,24cyclictest16950irq/224-800000007:24:550
3027526995814,18cyclictest16850irq/134-800000011:10:370
302752699581,33cyclictest3046948-21ssh09:37:560
3027534995735,7cyclictest3017393-21kworker/2:308:01:562
3027534995719,10cyclictest30-21ksoftirqd/211:05:162
3027529995711,7cyclictest23-21ksoftirqd/110:03:311
302752699574,34cyclictest131rcu_preempt10:50:020
3027536995643,7cyclictest3027974-21sendmail07:20:003
302752999565,6cyclictest23-21ksoftirqd/107:15:021
302752999565,10cyclictest23-21ksoftirqd/110:25:021
3027529995614,11cyclictest23-21ksoftirqd/108:20:021
3027529995613,6cyclictest0-21swapper/109:12:281
3027534995547,5cyclictest3017393-21kworker/2:309:32:162
3027534995538,5cyclictest3080878-21kworker/2:111:18:342
3027534995513,6cyclictest131rcu_preempt11:15:022
302752999556,6cyclictest23-21ksoftirqd/109:28:541
302752999556,6cyclictest23-21ksoftirqd/109:28:541
3027534995445,6cyclictest3044874-21kworker/2:010:28:122
302752999548,7cyclictest23-21ksoftirqd/112:01:261
302752999546,8cyclictest23-21ksoftirqd/112:07:021
302752999546,10cyclictest23-21ksoftirqd/111:40:011
302752999544,15cyclictest23-21ksoftirqd/110:59:081
302752999542,12cyclictest23-21ksoftirqd/111:50:321
3027526995426,4cyclictest16950irq/224-800000008:00:020
3027526995426,4cyclictest16950irq/224-800000008:00:010
3027526995425,5cyclictest309-21in:imuxsock08:55:000
3027534995344,6cyclictest3044874-21kworker/2:010:54:012
3027534995344,6cyclictest3044874-21kworker/2:010:54:012
3027534995343,5cyclictest3080878-21kworker/2:111:52:142
3027534995338,12cyclictest3033165-21systemctl09:00:002
302752999536,7cyclictest23-21ksoftirqd/110:13:421
302752999536,11cyclictest23-21ksoftirqd/111:30:001
302752999534,7cyclictest23-21ksoftirqd/109:18:001
3027529995342,7cyclictest23-21ksoftirqd/108:10:001
302753499526,6cyclictest131rcu_preempt09:25:112
302753499526,6cyclictest131rcu_preempt09:25:112
3027534995244,5cyclictest3109759-21kworker/2:212:20:002
3027534995244,5cyclictest3080878-21kworker/2:112:32:462
3027534995244,5cyclictest3080878-21kworker/2:111:34:502
3027534995244,5cyclictest3044874-21kworker/2:010:35:312
3027534995244,5cyclictest3033449-21kworker/2:209:19:342
3027534995243,6cyclictest3044874-21kworker/2:010:33:252
3027534995243,6cyclictest3044874-21kworker/2:009:51:292
3027534995243,5cyclictest3044874-21kworker/2:011:01:442
3027534995241,8cyclictest3109759-21kworker/2:212:13:582
3027534995241,6cyclictest3017393-21kworker/2:308:39:592
3027534995241,6cyclictest3017393-21kworker/2:308:39:582
3027534995232,7cyclictest3044874-21kworker/2:009:43:222
302752999527,9cyclictest23-21ksoftirqd/110:53:081
302752999527,9cyclictest23-21ksoftirqd/110:53:071
302752999522,12cyclictest23-21ksoftirqd/110:47:151
3027529995217,26cyclictest275-21dbus-daemon07:50:011
3027529995213,9cyclictest23-21ksoftirqd/111:48:331
3027529995210,4cyclictest131rcu_preempt11:30:461
3027526995223,7cyclictest12-21ksoftirqd/007:30:020
3027534995143,5cyclictest3123508-21kworker/2:312:27:452
3027534995143,5cyclictest3080878-21kworker/2:111:35:142
3027534995143,5cyclictest3044874-21kworker/2:010:41:302
3027534995142,6cyclictest3109759-21kworker/2:212:09:312
3027534995142,6cyclictest3109759-21kworker/2:211:55:042
3027534995142,6cyclictest3044874-21kworker/2:010:15:222
3027534995112,6cyclictest131rcu_preempt10:10:542
3027534995111,9cyclictest131rcu_preempt09:10:212
3027534995111,6cyclictest131rcu_preempt11:49:232
302752999517,8cyclictest23-21ksoftirqd/112:29:031
302752999516,9cyclictest23-21ksoftirqd/112:30:571
302752999515,9cyclictest23-21ksoftirqd/111:01:201
302752999514,16cyclictest23-21ksoftirqd/108:00:001
302752999514,16cyclictest23-21ksoftirqd/108:00:001
302752999514,11cyclictest23-21ksoftirqd/110:00:001
302752999513,10cyclictest23-21ksoftirqd/112:39:551
3027526995118,10cyclictest1-21systemd08:50:000
302752699511,30cyclictest0-21swapper/008:32:330
3027536995020,11cyclictest37-21ksoftirqd/311:05:003
3027534995043,5cyclictest3080878-21kworker/2:112:03:052
3027534995042,5cyclictest3109759-21kworker/2:212:21:182
3027534995042,5cyclictest3080878-21kworker/2:111:24:072
3027534995042,5cyclictest3080878-21kworker/2:110:56:402
3027534995042,5cyclictest3033449-21kworker/2:209:20:412
3027534995042,5cyclictest3017393-21kworker/2:310:07:432
3027534995042,5cyclictest3017393-21kworker/2:310:07:422
302752999504,13cyclictest23-21ksoftirqd/109:45:161
3027529995035,11cyclictest3467602-21systemd-journal08:22:551
302752999503,10cyclictest23-21ksoftirqd/109:31:171
302752999502,6cyclictest23-21ksoftirqd/109:37:551
3027526995032,14cyclictest3032122-21cat08:40:000
3027526995032,14cyclictest3032122-21cat08:40:000
3027536994919,11cyclictest37-21ksoftirqd/309:40:013
3027534994942,4cyclictest3017393-21kworker/2:310:20:172
3027534994941,5cyclictest3044874-21kworker/2:009:49:442
3027534994941,5cyclictest3044874-21kworker/2:009:37:052
3027534994941,5cyclictest3017393-21kworker/2:309:57:102
3027534994940,6cyclictest3080878-21kworker/2:112:36:362
3027534994939,7cyclictest3017393-21kworker/2:310:02:442
302752999494,10cyclictest23-21ksoftirqd/109:23:531
3027529994932,12cyclictest1-21systemd10:30:001
302752999492,11cyclictest3031863-21latency_hist08:35:021
3027529994915,7cyclictest0-21swapper/107:20:011
3027529994910,10cyclictest23-21ksoftirqd/109:53:201
3027536994831,13cyclictest3467602-21systemd-journal10:25:003
3027534994840,5cyclictest3017393-21kworker/2:308:23:272
3027534994839,6cyclictest3017393-21kworker/2:308:31:432
302752999487,7cyclictest23-21ksoftirqd/110:08:541
302752999487,7cyclictest23-21ksoftirqd/110:08:531
302752999486,11cyclictest23-21ksoftirqd/111:20:561
302752999484,9cyclictest23-21ksoftirqd/110:33:131
302752999482,8cyclictest3094039-21ssh11:17:461
302752999481,9cyclictest23-21ksoftirqd/109:43:101
302752999481,8cyclictest3029015-21munin-run07:40:001
3027526994841,4cyclictest16950irq/224-800000008:45:000
3027534994740,4cyclictest3021093-21kworker/2:007:31:412
3027534994740,4cyclictest3017393-21kworker/2:308:53:282
3027534994739,5cyclictest3021093-21kworker/2:007:14:532
3027534994739,5cyclictest3017393-21kworker/2:308:25:082
3027534994739,5cyclictest3017393-21kworker/2:307:53:352
3027534994731,12cyclictest3031073-21cron08:20:002
3027536994612,9cyclictest131rcu_preempt11:50:003
3027534994639,5cyclictest3021093-21kworker/2:007:25:082
3027534994639,4cyclictest3017393-21kworker/2:309:00:072
3027534994638,5cyclictest3017393-21kworker/2:308:45:402
3027534994638,5cyclictest3017393-21kworker/2:308:05:532
3027534994631,12cyclictest3467602-21systemd-journal07:35:032
302753699456,4cyclictest131rcu_preempt11:25:013
302753699455,35cyclictest150-21pr/ttyS209:57:023
3027536994534,6cyclictest309-21in:imuxsock12:35:023
3027536994523,16cyclictest275-21dbus-daemon10:35:013
3027536994513,11cyclictest37-21ksoftirqd/309:50:013
3027534994538,4cyclictest3017393-21kworker/2:307:41:122
3027534994537,5cyclictest3017393-21kworker/2:307:59:462
3027534994537,5cyclictest3017393-21kworker/2:307:59:462
3027534994537,5cyclictest3017393-21kworker/2:307:45:552
3027529994511,7cyclictest0-21swapper/108:05:001
302752999451,14cyclictest23-21ksoftirqd/108:45:011
3027526994534,8cyclictest3033157-21sendmail09:00:010
3027536994434,6cyclictest3121107-21munin-plugin-st12:15:003
3027536994430,10cyclictest3095023-21cat11:20:003
3027536994429,13cyclictest37-21ksoftirqd/309:18:323
302753699441,5cyclictest131rcu_preempt10:05:023
3027534994436,5cyclictest3021093-21kworker/2:007:18:112
3027534994436,5cyclictest3017393-21kworker/2:309:09:242
302752999446,9cyclictest0-21swapper/108:46:141
3027529994411,7cyclictest0-21swapper/109:00:001
3027526994427,13cyclictest3467602-21systemd-journal07:40:000
3027536994328,11cyclictest37-21ksoftirqd/309:31:183
302753699432,3cyclictest131rcu_preempt12:38:163
3027536994316,9cyclictest37-21ksoftirqd/311:00:003
3027536994316,21cyclictest131rcu_preempt12:09:203
3027534994336,4cyclictest3021093-21kworker/2:007:23:252
302752999436,6cyclictest23-21ksoftirqd/108:30:011
302753699428,18cyclictest131rcu_preempt10:50:513
302753699428,18cyclictest131rcu_preempt10:50:513
3027536994218,6cyclictest37-21ksoftirqd/311:07:213
3027536994218,20cyclictest361rcuc/310:30:023
302753699421,3cyclictest131rcu_preempt09:14:023
3027536994211,13cyclictest37-21ksoftirqd/311:45:023
3027534994234,5cyclictest3017393-21kworker/2:308:40:202
302752999426,7cyclictest0-21swapper/109:05:551
302752999421,11cyclictest23-21ksoftirqd/108:15:021
302752999421,11cyclictest23-21ksoftirqd/108:15:011
3027536994133,4cyclictest3030315-21sh08:05:003
3027536994131,7cyclictest37-21ksoftirqd/307:13:213
3027536994129,8cyclictest37-21ksoftirqd/310:10:003
3027536994129,8cyclictest37-21ksoftirqd/310:10:003
3027536994127,11cyclictest37-21ksoftirqd/312:16:303
3027536994126,9cyclictest3030048-21run-parts08:00:003
3027536994126,9cyclictest3030048-21run-parts08:00:003
3027536994125,12cyclictest3064301-21systemd-run10:15:013
3027536994115,15cyclictest37-21ksoftirqd/312:24:273
302753699411,20cyclictest0-21swapper/311:55:033
3027529994132,6cyclictest23-21ksoftirqd/107:30:021
3027529994127,10cyclictest23-21ksoftirqd/108:40:011
3027529994127,10cyclictest23-21ksoftirqd/108:40:001
302752999412,10cyclictest23-21ksoftirqd/107:45:011
3027526994126,11cyclictest3467602-21systemd-journal08:05:000
302753699406,5cyclictest131rcu_preempt08:15:003
302753699406,5cyclictest131rcu_preempt08:15:003
3027536994028,9cyclictest37-21ksoftirqd/310:15:163
3027536994027,9cyclictest1-21systemd07:45:013
3027536994026,10cyclictest37-21ksoftirqd/309:00:013
302753699402,5cyclictest131rcu_preempt10:45:013
3027536994024,12cyclictest37-21ksoftirqd/309:25:013
302753699401,6cyclictest131rcu_preempt11:15:003
302753699401,5cyclictest3081011-21ssh10:50:023
3027526994034,4cyclictest0-21swapper/007:10:020
3027526994023,13cyclictest3030804-21sshd08:14:140
3027526994023,13cyclictest3030804-21sshd08:14:140
3027536993925,11cyclictest37-21ksoftirqd/307:55:003
3027536993925,10cyclictest37-21ksoftirqd/312:27:293
3027536993916,9cyclictest37-21ksoftirqd/311:36:033
3027536993915,4cyclictest37-21ksoftirqd/310:36:013
3027536993913,7cyclictest131rcu_preempt12:03:553
3027536993911,18cyclictest0-21swapper/309:44:203
3027536993911,16cyclictest37-21ksoftirqd/307:30:023
302752999398,6cyclictest0-21swapper/107:52:371
302752999391,4cyclictest0-21swapper/107:33:311
302753699383,3cyclictest131rcu_preempt11:33:293
302753699381,8cyclictest0-21swapper/309:27:003
302753699381,8cyclictest0-21swapper/309:26:593
3027536993814,7cyclictest37-21ksoftirqd/309:10:023
302752699381,5cyclictest3031352-21cron08:25:000
302753699377,6cyclictest131rcu_preempt11:57:173
302753699376,6cyclictest131rcu_preempt08:40:003
302753699376,6cyclictest131rcu_preempt08:40:003
3027536993728,5cyclictest309-21in:imuxsock08:44:183
3027536993726,7cyclictest37-21ksoftirqd/308:35:003
3027536993724,10cyclictest37-21ksoftirqd/307:35:023
3027536993712,18cyclictest37-21ksoftirqd/308:25:013
3027536993711,18cyclictest37-21ksoftirqd/309:53:023
302752699371,9cyclictest1-21systemd07:50:550
302752699371,32cyclictest0-21swapper/008:05:020
302753699363,4cyclictest131rcu_preempt11:25:043
3027536993623,10cyclictest3032874-21sshd08:52:553
302753699361,5cyclictest131rcu_preempt08:50:013
302753699352,5cyclictest131rcu_preempt08:09:553
302752999351,31cyclictest23-21ksoftirqd/108:53:471
3027526993528,5cyclictest3031219-21latency_hist08:20:010
302752699352,4cyclictest3033419-21sshd09:04:550
3027536993425,7cyclictest37-21ksoftirqd/307:40:013
3027536993423,8cyclictest37-21ksoftirqd/307:45:563
3027536993423,8cyclictest3033407-21sshd09:02:553
3027536993423,8cyclictest3027574-21sed07:10:033
302752699341,29cyclictest3029255-21kworker/u8:307:47:380
302753699331,6cyclictest131rcu_preempt08:20:013
3027529993325,5cyclictest3027650-21latency_hist07:10:021
302753699327,6cyclictest131rcu_preempt08:27:553
3027536993222,7cyclictest37-21ksoftirqd/307:21:553
302753499261,6cyclictest131rcu_preempt07:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional