You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-11 - 02:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Thu Dec 11, 2025 00:39:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2823790996452,8cyclictest2912083-21ssh22:20:012
2823788995852,3cyclictest0-21swapper/000:19:240
2823789995550,2cyclictest0-21swapper/100:04:281
2823789995330,18cyclictest0-21swapper/122:04:231
2823789995312,15cyclictest422-21dbus-daemon21:10:011
282378999503,20cyclictest3010353-21ssh00:34:231
2823788995022,19cyclictest2919831-21ssh22:30:130
2823788995020,15cyclictest2891172-21ssh21:52:420
2823788995018,8cyclictest2896301-21ntpq21:59:260
2823789994925,21cyclictest3007369-21timerwakeupswit00:29:351
282378899493,39cyclictest1055298-21cups-browsed23:24:230
2823788994919,10cyclictest2877621-21ssh21:34:240
2823788994918,12cyclictest2853764-21munin-run20:50:010
282378999481,20cyclictest3013805-21ssh00:39:251
2823788994822,7cyclictest2854707-21ntpq20:54:250
2823788994819,12cyclictest2840786-21df_inode20:09:140
2823788994817,10cyclictest3013752-21ntpq00:39:230
282378999473,16cyclictest2952073-21ssh23:14:251
282378999473,16cyclictest2952073-21ssh23:14:251
2823789994724,20cyclictest0-21swapper/123:20:011
2823789994718,2cyclictest0-21swapper/122:39:261
282378999471,17cyclictest2826183-21sshd19:19:211
2823788994740,4cyclictest151ktimers/020:29:300
2823788994740,4cyclictest151ktimers/020:29:300
2823788994729,13cyclictest3003686-21sshd00:24:250
2823788994721,11cyclictest2834838-21gltestperf19:49:190
2823788994717,9cyclictest2850800-21systemctl20:40:010
2823789994636,6cyclictest2997737-21latency_hist00:15:021
2823789994636,6cyclictest2997737-21latency_hist00:15:021
2823789994629,13cyclictest0-21swapper/100:10:011
2823789994629,13cyclictest0-21swapper/100:10:011
2823789994624,18cyclictest0-21swapper/123:04:241
282378999462,17cyclictest2896310-21ssh21:59:261
2823789994621,21cyclictest0-21swapper/100:00:031
2823788994629,11cyclictest2827730-21ntpq19:24:250
2823788994627,14cyclictest12450irq/32-eth023:29:230
2823788994619,8cyclictest2845347-21/usr/sbin/munin20:24:150
2823790994537,5cyclictest2930003-21grep22:44:282
2823790994537,5cyclictest2930003-21grep22:44:282
282378999454,15cyclictest2948280-21ssh23:09:231
282378999454,15cyclictest2948280-21ssh23:09:231
2823789994520,21cyclictest0-21swapper/122:44:241
2823789994520,21cyclictest0-21swapper/122:44:241
2823788994519,7cyclictest2924482-21ssh22:37:170
2823788994518,8cyclictest2942630-21ssh23:01:260
2823788994517,9cyclictest2906154-21sh22:12:540
2823788994517,16cyclictest2996262-21ssh00:14:150
2823788994517,16cyclictest2996262-21ssh00:14:150
2823790994437,2cyclictest0-21swapper/200:24:252
2823789994437,4cyclictest2850811-21latency_hist20:40:011
2823789994436,4cyclictest0-21swapper/121:29:251
2823789994423,17cyclictest0-21swapper/121:24:261
2823789994422,19cyclictest0-21swapper/122:34:341
282378999442,16cyclictest2888573-21irqstats21:49:151
2823789994419,2cyclictest0-21swapper/123:24:241
2823788994422,8cyclictest2951991-21ntpq23:14:220
2823788994422,8cyclictest2951991-21ntpq23:14:220
2823788994420,17cyclictest0-21swapper/019:39:220
2823788994419,8cyclictest3010326-21ntpq00:34:220
2823788994418,9cyclictest2909916-21sh22:17:580
2823788994418,7cyclictest2833658-21ntpq19:44:240
2823788994418,7cyclictest2833658-21ntpq19:44:240
2823788994418,14cyclictest2855671-21aten2.4-expect20:59:150
2823788994417,9cyclictest2985588-21ssh23:59:220
2823788994417,9cyclictest2982948-21cat23:55:030
2823788994417,7cyclictest2937330-21ntpq22:54:270
2823788994417,7cyclictest2937330-21ntpq22:54:270
2823788994416,8cyclictest2888943-21ntpq21:49:230
282379099432,32cyclictest2959443-21ssh23:24:242
2823789994335,5cyclictest2911843-21head22:19:351
2823789994322,17cyclictest0-21swapper/123:54:241
282378999432,18cyclictest2892602-21sh21:54:271
282378999431,17cyclictest2966911-21ntpq23:34:231
2823788994320,6cyclictest2884005-21sh21:43:160
2823788994318,7cyclictest2861829-21sh21:13:300
2823788994317,8cyclictest2867610-21latency_hist21:20:030
2823788994317,8cyclictest2867610-21latency_hist21:20:030
2823788994317,10cyclictest2903800-21ntpq22:09:250
2823788994316,9cyclictest2917496-21rm22:28:130
2823790994235,5cyclictest0-21swapper/223:40:012
2823790994228,10cyclictest2934044-21timerandwakeup22:49:352
282378999424,11cyclictest2941012-21ssh22:59:301
282378999422,11cyclictest1055298-21cups-browsed23:29:221
2823789994219,3cyclictest0-21swapper/121:14:341
2823788994220,5cyclictest2970660-21ntpq23:39:220
2823788994218,8cyclictest2848650-21ntpq20:34:230
2823788994218,7cyclictest2843225-21sed20:14:340
2823788994217,7cyclictest2879936-21sh21:37:380
2823788994217,14cyclictest2938443-21ssh22:55:140
2823788994217,10cyclictest3005672-21sh00:27:290
2823788994217,10cyclictest2989170-21ssh00:04:190
2823788994216,10cyclictest2974699-21sendmail_mailst23:44:310
2823788994214,9cyclictest0-21swapper/000:09:140
2823788994214,9cyclictest0-21swapper/000:09:140
2823788994214,16cyclictest2976204-21ssh23:46:270
2823790994135,3cyclictest0-21swapper/223:19:362
2823790994132,4cyclictest0-21swapper/222:24:242
2823790994132,3cyclictest0-21swapper/223:00:012
282378999414,13cyclictest2918720-21ntpq22:29:261
2823789994134,5cyclictest2885737-21unixbench_multi21:44:371
2823789994133,5cyclictest422-21dbus-daemon19:40:011
2823789994132,6cyclictest421-21avahi-daemon21:40:011
282378999412,15cyclictest2977701-21df_inode23:49:131
2823789994117,3cyclictest0-21swapper/100:24:251
2823788994129,5cyclictest959-21mta-sts-daemon23:19:240
2823788994118,7cyclictest2836630-21ntpq19:54:180
2823788994118,7cyclictest2826835-21cut19:20:020
2823788994117,9cyclictest2858180-21tail21:04:350
2823788994117,6cyclictest2844217-21ntpq20:19:250
2823788994116,9cyclictest2869366-21rm21:23:500
2823790994030,7cyclictest453-21systemd-logind21:05:022
2823789994030,7cyclictest591-21sshd23:44:241
282378999402,15cyclictest2937278-21ssh22:54:251
282378999402,15cyclictest2937278-21ssh22:54:251
282378999401,15cyclictest2829188-21ntpq19:29:241
2823788994017,7cyclictest2946638-21sh23:07:100
2823788994017,7cyclictest2946638-21sh23:07:100
2823788994016,8cyclictest2965816-21sh23:33:250
2823788994016,7cyclictest2927484-21rm22:40:430
2823788994016,7cyclictest2927484-21rm22:40:430
2823788994016,6cyclictest2859766-21diskmemload22:02:450
2823788994016,6cyclictest2851044-21hostname20:44:110
2823788994016,10cyclictest2824473-21latency19:14:190
2823788994015,9cyclictest2934578-21cat22:50:020
2823788994015,9cyclictest2829701-21awk19:29:350
2823788994015,14cyclictest2912584-21ssh22:20:340
2823790993931,5cyclictest2927000-21latency_hist22:40:022
2823790993926,10cyclictest2885015-21missed_timers21:44:222
2823790993921,13cyclictest422-21dbus-daemon21:39:242
282378999393,9cyclictest2907508-21ssh22:14:271
2823789993919,5cyclictest2878286-21unixbench_singl21:34:371
2823789993916,5cyclictest2856579-21timerwakeupswit20:59:351
2823788993914,7cyclictest2873353-21aten_r2power_po21:29:130
2823790993831,4cyclictest2997584-21debian-sa100:15:002
2823790993831,4cyclictest2997584-21debian-sa100:15:002
2823790993827,6cyclictest0-21swapper/221:54:252
2823789993829,3cyclictest0-21swapper/121:19:241
2823789993829,3cyclictest0-21swapper/121:19:241
2823788993815,7cyclictest2839873-21run-parts20:04:310
2823790993729,5cyclictest2832817-21latency_hist19:40:022
2823790993727,7cyclictest2847802-21latency_hist20:30:022
2823790993727,7cyclictest2847802-21latency_hist20:30:022
2823790993724,9cyclictest2985524-21needreboot23:59:202
2823790993724,5cyclictest2859971-21latency_hist21:10:022
2823789993729,5cyclictest2933501-21needreboot22:49:211
282378999372,13cyclictest3000360-21ssh00:19:271
2823788993728,4cyclictest2830770-21ntpq19:34:290
2823788993725,8cyclictest2859314-21ps21:09:280
2823790993628,5cyclictest2837260-21sh19:55:012
2823790993626,6cyclictest2982314-21xargs23:54:362
282378999364,9cyclictest2970723-21ssh23:39:241
2823789993629,4cyclictest2844972-21latency_hist20:20:031
2823789993625,8cyclictest2841103-21needreboot20:09:201
2823788993623,10cyclictest2838173-21ntpq19:59:240
2823790993521,11cyclictest2919059-21switchtime22:29:362
2823790993520,9cyclictest2900191-21sh22:04:322
282378999352,14cyclictest2914528-21fschecks_count22:24:161
2823790993427,4cyclictest2849295-21latency_hist20:35:012
2823790993426,5cyclictest3001102-21munin-run00:20:012
2823790993426,5cyclictest2825460-21latency_hist19:15:032
2823790993417,4cyclictest2823787-21cyclictest23:09:272
2823790993417,4cyclictest2823787-21cyclictest23:09:272
282379099341,25cyclictest2937358-21ssh22:54:282
282379099341,25cyclictest2937358-21ssh22:54:282
2823789993425,6cyclictest421-21avahi-daemon22:10:021
2823789993423,8cyclictest2833675-21ntp_kernel_pll_19:44:251
2823789993423,8cyclictest2833675-21ntp_kernel_pll_19:44:251
2823789993415,15cyclictest422-21dbus-daemon19:59:251
282379099334,3cyclictest171rcu_preempt23:04:262
2823790993328,2cyclictest0-21swapper/221:15:012
2823790993326,4cyclictest2895864-21ssh21:59:172
2823790993326,3cyclictest0-21swapper/223:26:462
2823790993323,7cyclictest491-21gdbus00:39:332
2823790993322,8cyclictest3006847-21ntp_kernel_err00:29:222
2823789993325,5cyclictest421-21avahi-daemon20:44:291
2823789993325,5cyclictest2840280-21munin-plugin-st20:05:011
282378999332,24cyclictest2853132-21sshd20:49:211
2823788993320,9cyclictest2823801-21unin-run19:10:020
2823789993224,5cyclictest2855279-21systemd-run20:55:021
2823789993222,7cyclictest2825069-21timerwakeupswit19:14:341
2823790993125,3cyclictest57-21kcompactd023:49:232
2823790993124,3cyclictest0-21swapper/200:34:312
282379099312,3cyclictest171rcu_preempt23:14:232
282379099312,3cyclictest171rcu_preempt23:14:232
2823790993123,5cyclictest453-21systemd-logind20:20:012
2823790993123,5cyclictest2989914-21unixbench_multi00:04:342
2823790993123,5cyclictest2834975-21latency19:49:202
2823790993121,7cyclictest2888021-21sh21:48:562
2823789993123,5cyclictest2830128-21aten_r2power_fr19:34:131
282379099303,21cyclictest2922491-21ssh22:34:282
2823790993024,3cyclictest0-21swapper/222:15:012
2823790993024,3cyclictest0-21swapper/220:55:012
2823790993021,6cyclictest1055298-21cups-browsed22:05:382
282379099293,20cyclictest2966916-21ssh23:34:232
2823790992924,2cyclictest0-21swapper/219:25:012
2823790992923,3cyclictest0-21swapper/223:44:202
2823790992922,4cyclictest2871105-21cron21:25:012
2823790992922,4cyclictest2826774-21munin-run19:20:012
2823790992921,5cyclictest959-21mta-sts-daemon20:45:502
2823790992921,5cyclictest2878526-21systemd-run21:35:012
2823789992918,8cyclictest2847168-21ntp_kernel_pll_20:29:251
2823789992918,8cyclictest2847168-21ntp_kernel_pll_20:29:251
2823790992822,3cyclictest0-21swapper/220:45:022
2823790992821,3cyclictest0-21swapper/221:19:352
2823790992821,3cyclictest0-21swapper/221:19:352
2823790992820,5cyclictest2831207-21unixbench_singl19:34:362
2823790992820,4cyclictest698-21cron20:25:012
2823790992815,10cyclictest266-21systemd-journal21:29:232
2823789992821,4cyclictest0-21swapper/120:35:001
282379099272,6cyclictest0-21swapper/220:59:182
2823790992719,5cyclictest422-21dbus-daemon19:25:192
2823789992720,3cyclictest0-21swapper/119:25:011
2823789992719,5cyclictest2836327-21gltestperf19:54:141
282378999271,23cyclictest956-21mta-sts-daemon19:48:051
282379199262,6cyclictest0-21swapper/321:09:243
2823790992618,5cyclictest2991419-21ssh00:07:142
2823790992618,5cyclictest2991419-21ssh00:07:142
2823790992617,6cyclictest421-21avahi-daemon20:36:402
282378999264,16cyclictest1690079-21ntpd21:04:301
2823789992621,2cyclictest0-21swapper/120:14:331
2823789992620,3cyclictest0-21swapper/119:10:021
2823790992519,3cyclictest0-21swapper/219:44:232
2823790992519,3cyclictest0-21swapper/219:44:232
2823790992519,2cyclictest0-21swapper/220:00:012
2823790992518,4cyclictest2839440-21kernelversion20:04:192
2823790992517,5cyclictest1055298-21cups-browsed20:06:122
2823790992514,8cyclictest2843048-21timerandwakeup20:14:332
282379199243,5cyclictest0-21swapper/321:29:273
282379199242,5cyclictest0-21swapper/319:24:243
282379199241,5cyclictest0-21swapper/321:39:263
282379199233,6cyclictest0-21swapper/322:39:223
282379199232,5cyclictest0-21swapper/323:54:283
282379199223,8cyclictest0-21swapper/300:04:283
282379199223,5cyclictest0-21swapper/300:09:243
282379199223,5cyclictest0-21swapper/300:09:243
282379199222,6cyclictest0-21swapper/322:29:313
282379199222,5cyclictest0-21swapper/322:34:223
282379199221,6cyclictest0-21swapper/323:59:223
282379199211,6cyclictest0-21swapper/322:09:233
282379199211,6cyclictest0-21swapper/321:24:263
282379199211,5cyclictest0-21swapper/323:44:263
282379099218,9cyclictest422-21dbus-daemon19:10:012
282378999211,15cyclictest2845857-21ps20:24:281
282379199203,9cyclictest0-21swapper/300:29:303
282379199203,5cyclictest0-21swapper/323:14:243
282379199203,5cyclictest0-21swapper/323:14:243
282379199202,7cyclictest0-21swapper/319:19:253
282379199202,6cyclictest0-21swapper/321:59:253
282379199202,5cyclictest0-21swapper/322:44:213
282379199202,5cyclictest0-21swapper/322:44:213
282379199202,5cyclictest0-21swapper/321:44:303
282379199202,5cyclictest0-21swapper/300:39:243
282379199201,6cyclictest0-21swapper/323:39:233
282379199201,6cyclictest0-21swapper/322:10:233
282379199192,7cyclictest0-21swapper/322:59:333
282379199192,5cyclictest0-21swapper/323:09:233
282379199192,5cyclictest0-21swapper/323:09:233
282379199192,5cyclictest0-21swapper/323:04:193
282379199192,5cyclictest0-21swapper/321:34:303
282379199191,5cyclictest0-21swapper/300:19:253
282379199190,7cyclictest0-21swapper/323:19:223
282379199182,5cyclictest0-21swapper/322:54:283
282379199182,5cyclictest0-21swapper/322:54:283
282379199173,7cyclictest0-21swapper/300:14:253
282379199173,7cyclictest0-21swapper/300:14:253
282379199171,4cyclictest0-21swapper/321:15:013
282379199163,9cyclictest0-21swapper/320:54:283
282379199163,7cyclictest0-21swapper/320:29:293
282379199163,7cyclictest0-21swapper/320:29:293
282379199162,6cyclictest0-21swapper/319:54:223
282379199152,7cyclictest0-21swapper/322:19:263
282379199152,7cyclictest0-21swapper/322:04:223
282379199152,7cyclictest0-21swapper/321:49:223
282379199152,7cyclictest0-21swapper/300:34:233
282379199152,6cyclictest0-21swapper/320:39:223
282379199142,7cyclictest0-21swapper/321:19:243
282379199142,7cyclictest0-21swapper/321:19:243
282379199142,5cyclictest0-21swapper/320:19:213
282379199141,8cyclictest0-21swapper/323:34:253
282379199133,6cyclictest0-21swapper/323:29:233
282379199123,7cyclictest0-21swapper/322:49:123
282379199123,6cyclictest0-21swapper/320:49:213
282379199123,6cyclictest0-21swapper/319:59:253
282379199122,6cyclictest0-21swapper/322:24:173
282379199122,6cyclictest0-21swapper/300:24:233
282379199114,5cyclictest0-21swapper/320:59:243
282379199113,5cyclictest0-21swapper/321:54:133
282379199113,5cyclictest0-21swapper/320:09:233
282379199111,7cyclictest0-21swapper/323:49:153
282379199101,7cyclictest0-21swapper/323:24:253
282379199101,4cyclictest0-21swapper/320:44:233
28237919992,5cyclictest0-21swapper/321:04:233
28237919992,5cyclictest0-21swapper/319:39:263
28237919991,6cyclictest0-21swapper/320:14:233
28237919991,6cyclictest0-21swapper/319:34:283
28237919991,6cyclictest0-21swapper/319:15:013
28237919991,4cyclictest0-21swapper/319:29:283
28237919982,4cyclictest0-21swapper/320:34:233
28237919981,6cyclictest0-21swapper/320:24:233
28237919981,6cyclictest0-21swapper/320:04:263
28237919981,6cyclictest0-21swapper/319:10:013
28237919981,5cyclictest0-21swapper/319:49:233
28237919971,4cyclictest0-21swapper/319:44:153
28237919971,4cyclictest0-21swapper/319:44:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional