You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 17:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot5s.osadl.org (updated Wed Nov 26, 2025 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2710692995332,17cyclictest2776444-21sh10:20:011
2710692995128,19cyclictest0-21swapper/112:40:011
2710688995039,6cyclictest2793035-21grep11:05:240
2710688995020,24cyclictest0-21swapper/010:20:010
2710692994822,21cyclictest0-21swapper/108:40:191
2710688994836,8cyclictest2748945-21cut09:00:000
2710692994716,3cyclictest0-21swapper/112:05:251
2710702994614,12cyclictest2718961-21ls07:30:203
2710688994615,28cyclictest2752361-21idleruntime-cro09:10:010
2710688994615,28cyclictest2752361-21idleruntime-cro09:10:000
2710692994513,5cyclictest2790096-21sendmail11:00:001
2710688994517,24cyclictest0-21swapper/007:50:400
2710698994437,4cyclictest2800936-21fschecks_count11:30:142
2710692994437,4cyclictest2800543-21latency_hist11:30:001
2710692994433,7cyclictest2793038-21unixbench_multi11:05:241
2710692994415,4cyclictest0-21swapper/108:20:011
2710688994417,22cyclictest0-21swapper/010:05:000
2710688994414,3cyclictest2786679-21sh10:50:000
2710688994413,8cyclictest215-21dbus-daemon11:40:060
2710692994333,6cyclictest2736572-21unixbench_singl08:20:271
2710692994317,3cyclictest0-21swapper/107:20:191
2710692994315,3cyclictest0-21swapper/111:40:241
2710692994312,6cyclictest0-21swapper/112:05:001
271068899431,7cyclictest2790262-21latency_hist11:00:020
271068899431,13cyclictest2819297-21cron12:25:000
2710688994310,6cyclictest2758983-21grep09:25:290
2710702994227,10cyclictest2716869-21/usr/sbin/munin07:25:123
2710698994235,5cyclictest2755960-21kworker/u8:0+events_unbound09:24:392
271069299422,13cyclictest2754227-21latency_hist09:15:001
271068899422,11cyclictest2782832-21unixbench_singl10:35:270
2710688994212,4cyclictest2817595-21sendmail12:20:000
2710702994131,6cyclictest2775187-21fschecks_time10:15:163
2710702994128,10cyclictest2823627-21/usr/sbin/munin12:35:223
2710702994127,10cyclictest2725740-21/usr/sbin/munin07:50:193
2710702994113,12cyclictest2752669-21cat09:10:153
2710702994111,12cyclictest2754484-21rm09:15:023
2710698994135,4cyclictest2767537-21kworker/u8:2+events_unbound10:45:022
2710698994121,6cyclictest131rcu_preempt10:50:112
2710692994127,8cyclictest2784530-21grep10:40:281
271068899418,11cyclictest2739120-21fschecks_count08:30:170
2710688994118,19cyclictest0-21swapper/009:45:270
2710688994116,21cyclictest0-21swapper/011:55:250
2710688994116,21cyclictest0-21swapper/011:55:250
2710702994026,11cyclictest2761909-21/usr/sbin/munin09:35:213
2710702994024,11cyclictest2799515-21/usr/sbin/munin11:25:203
2710698994033,5cyclictest2778123-21kworker/u8:0+events_unbound10:29:492
2710692994028,8cyclictest231ktimers/109:45:141
271068899402,7cyclictest2728854-21fschecks_count08:00:160
271068899402,11cyclictest2800076-21sed11:25:260
271068899401,9cyclictest2790317-21sshd11:00:010
2710688994016,21cyclictest13650irq/42-30be0000.ethernet09:50:140
2710688994013,3cyclictest0-21swapper/011:15:160
2710688994011,4cyclictest27110542sleep007:10:170
271068899401,12cyclictest2753630-21head09:10:270
271068899401,11cyclictest440-21cron08:20:010
2710702993927,9cyclictest2737782-21/usr/sbin/munin08:25:213
2710702993926,8cyclictest2770271-21/usr/sbin/munin10:00:183
2710702993925,11cyclictest2765095-21/usr/sbin/munin09:45:203
2710702993924,12cyclictest2758027-21/usr/sbin/munin09:25:173
2710702993924,10cyclictest2806055-21/usr/sbin/munin11:45:163
2710702993923,10cyclictest0-21swapper/311:35:123
2710702993921,13cyclictest0-21swapper/309:50:123
2710698993925,6cyclictest2798472-21kworker/u8:2+events_unbound11:25:442
2710692993930,5cyclictest2745535-21idleruntime-cro08:50:001
271068899394,8cyclictest2764660-21latency_hist09:45:010
271068899391,10cyclictest2801816-21grep11:30:240
271068899390,14cyclictest0-21swapper/010:50:220
2710702993828,7cyclictest2808161-21/usr/sbin/munin11:50:223
2710702993828,7cyclictest2795995-21/usr/sbin/munin11:15:183
2710702993827,8cyclictest2779504-21sshd10:25:483
2710702993827,7cyclictest2790463-21/usr/sbin/munin11:00:163
2710702993827,7cyclictest2730758-21/usr/sbin/munin08:05:193
2710702993826,9cyclictest2782473-21/usr/sbin/munin10:35:243
2710702993826,8cyclictest2729055-21/usr/sbin/munin08:00:193
2710702993826,7cyclictest2813060-21/usr/sbin/munin12:05:183
2710702993826,7cyclictest2739293-21/usr/sbin/munin08:30:193
2710702993823,10cyclictest2793856-21/usr/sbin/munin11:10:143
2710702993823,10cyclictest2722434-21/usr/sbin/munin07:40:223
2710702993818,4cyclictest2773047-21munin-run10:10:003
2710702993817,16cyclictest2751901-21tail09:05:253
2710702993817,16cyclictest2751901-21tail09:05:243
2710698993830,4cyclictest2791322-21grep11:00:252
2710692993830,4cyclictest0-21swapper/111:35:001
271068899388,5cyclictest0-21swapper/008:35:190
2710688993829,6cyclictest2745950-21fschecks_time08:50:150
2710688993827,7cyclictest2755998-21idleruntime-cro09:20:010
2710688993827,7cyclictest2743411-21grep08:40:280
271068899382,11cyclictest2803932-21cron11:40:000
271068899382,11cyclictest2721642-21apt07:40:010
2710688993812,5cyclictest121ktimers/009:20:190
271068899381,14cyclictest2716556-21idleruntime-cro07:25:010
271068899381,10cyclictest2797587-21sort11:20:160
2710702993727,7cyclictest2742708-21/usr/sbin/munin08:40:223
2710702993727,6cyclictest0-21swapper/308:25:013
2710702993726,7cyclictest2798499-21cron11:21:003
2710702993726,7cyclictest2792388-21/usr/sbin/munin11:05:173
2710702993725,9cyclictest2746135-21/usr/sbin/munin08:50:183
2710702993725,8cyclictest2763354-21/usr/sbin/munin09:40:183
2710702993725,7cyclictest2776694-21munin-node10:20:123
2710702993724,10cyclictest2816461-21/usr/sbin/munin12:15:203
2710702993724,10cyclictest2756571-21/usr/sbin/munin09:20:183
2710702993723,10cyclictest2774478-21cron10:12:013
2710702993710,9cyclictest2732851-21tr08:10:243
2710698993731,4cyclictest2736870-21kworker/u8:1+events_unbound08:24:322
2710692993729,4cyclictest0-21swapper/107:15:191
271068899379,6cyclictest2811264-21ls12:00:160
2710688993715,19cyclictest2717306-21ntpq07:25:190
271070299369,9cyclictest2784000-21missed_timers10:40:223
2710702993627,6cyclictest2721461-21cron07:38:003
2710702993626,7cyclictest2818182-21/usr/sbin/munin12:20:183
2710702993626,7cyclictest2809819-21/usr/sbin/munin11:55:213
2710702993626,7cyclictest2809819-21/usr/sbin/munin11:55:203
2710702993626,7cyclictest2804536-21/usr/sbin/munin11:40:193
2710702993626,7cyclictest2788960-21/usr/sbin/munin10:55:193
2710702993626,7cyclictest2768494-21/usr/sbin/munin09:55:183
2710702993626,7cyclictest2768494-21/usr/sbin/munin09:55:183
2710702993626,7cyclictest2744734-21/usr/sbin/munin08:45:243
2710702993626,7cyclictest2734157-21/usr/sbin/munin08:15:193
2710702993624,9cyclictest2820649-21sshd12:25:483
2710702993624,9cyclictest2781202-21sshd10:30:493
2710702993624,9cyclictest2727589-21/usr/sbin/munin07:55:233
2710702993622,9cyclictest2712091-21unixbench_singl07:10:283
2710702993620,12cyclictest2811857-21/usr/sbin/munin12:00:233
2710702993620,12cyclictest2724663-21/usr/sbin/munin07:45:263
2710702993620,12cyclictest2724663-21/usr/sbin/munin07:45:263
2710702993617,15cyclictest2740935-21if_eth008:35:163
2710698993627,6cyclictest2817600-21idleruntime-cro12:20:002
2710698993627,5cyclictest0-21swapper/207:15:012
271069299363,10cyclictest2758926-21unixbench_multi09:25:281
2710692993626,7cyclictest2783263-21sh10:40:001
2710692993625,7cyclictest2781092-21grep10:30:281
2710688993629,4cyclictest2761092-21cron09:35:000
271068899361,8cyclictest2745534-21latency_hist08:50:000
2710688993612,3cyclictest637-21runrttasks07:15:530
2710702993525,7cyclictest2822531-21cron12:33:003
2710702993525,7cyclictest2814415-21munin-node12:10:113
2710702993525,7cyclictest2801963-21sshd11:31:483
2710702993525,7cyclictest2760171-21/usr/sbin/munin09:30:213
2710702993525,7cyclictest2749526-21/usr/sbin/munin09:00:173
2710702993525,7cyclictest2715750-21/usr/sbin/munin07:20:253
2710702993525,7cyclictest2714477-21cron07:16:013
2710702993524,8cyclictest2786324-21sshd10:45:483
2710702993522,10cyclictest2787317-21/usr/sbin/munin10:50:183
2710698993527,4cyclictest0-21swapper/212:10:162
2710698993519,6cyclictest2721409-21kworker/u8:2+events_unbound07:45:032
2710698993519,6cyclictest2721409-21kworker/u8:2+events_unbound07:45:022
2710688993514,3cyclictest2710680-21cyclictest10:05:250
271068899351,12cyclictest2738701-21cron08:30:010
2710698993424,4cyclictest0-21swapper/212:05:252
2710698993419,6cyclictest2743659-21kworker/u8:1+events_unbound08:59:232
2710698993418,5cyclictest2757934-21kworker/u8:1+events_unbound09:50:552
2710692993426,5cyclictest0-21swapper/112:20:001
271068899346,5cyclictest2823990-21unixbench_singl12:35:240
2710688993411,4cyclictest121ktimers/011:10:150
2710688993411,3cyclictest13650irq/42-30be0000.ethernet10:10:220
271068899340,2cyclictest0-21swapper/009:55:270
271068899340,2cyclictest0-21swapper/009:55:260
2710698993325,6cyclictest2750524-21kworker/u8:2+events_unbound09:15:552
2710698993325,5cyclictest2817439-21kworker/u8:2+events_unbound12:30:142
2710698993324,6cyclictest2778123-21kworker/u8:0+events_unbound10:35:012
2710698993324,5cyclictest2812028-21tail12:00:242
271068899338,5cyclictest2805993-21expr11:45:140
2710688993322,8cyclictest2725070-21munin-plugin-st07:50:000
2710688993322,8cyclictest2725070-21munin-plugin-st07:50:000
2710688993313,3cyclictest0-21swapper/007:55:240
2710688993311,4cyclictest121ktimers/008:10:210
2710698993224,4cyclictest0-21swapper/207:25:232
2710698993224,4cyclictest0-21swapper/207:20:292
2710698993221,6cyclictest53-21kcompactd008:00:002
2710698993219,9cyclictest1-21systemd11:15:032
2710692993225,4cyclictest0-21swapper/107:35:251
2710692993224,4cyclictest0-21swapper/110:20:161
2710688993217,2cyclictest0-21swapper/010:40:340
2710698993123,4cyclictest0-21swapper/209:40:172
2710698993123,4cyclictest0-21swapper/209:35:152
2710698993121,6cyclictest0-21swapper/207:40:262
2710698993116,5cyclictest2798472-21kworker/u8:2+events_unbound11:39:352
2710692993123,4cyclictest0-21swapper/111:00:251
2710692993122,5cyclictest0-21swapper/108:10:251
2710688993123,4cyclictest0-21swapper/011:50:080
271068899310,14cyclictest0-21swapper/009:35:030
2710698993022,4cyclictest0-21swapper/210:05:002
2710698993020,6cyclictest2747244-21munin-run08:55:002
271069899301,8cyclictest0-21swapper/209:10:142
2710698993014,5cyclictest2773018-21kworker/u8:0+events_unbound10:11:282
2710692993022,4cyclictest0-21swapper/112:20:251
2710692993021,5cyclictest0-21swapper/111:15:261
2710688993011,4cyclictest121ktimers/012:25:190
2710698992923,4cyclictest2728360-21kworker/u8:0+events_unbound08:25:432
2710698992922,4cyclictest2824440-21idleruntime-cro12:40:002
2710698992921,6cyclictest2773018-21kworker/u8:0+events_unbound10:15:422
2710698992921,5cyclictest2726774-21munin-plugin-st07:55:002
2710698992919,8cyclictest2743844-21cut08:45:012
2710692992923,3cyclictest0-21swapper/111:45:251
2710692992922,4cyclictest0-21swapper/109:35:151
2710692992918,7cyclictest2821000-21cron12:30:001
2710692992916,9cyclictest0-21swapper/107:55:171
2710688992922,4cyclictest0-21swapper/009:00:230
271068899290,3cyclictest0-21swapper/007:32:110
2710698992820,5cyclictest0-21swapper/211:10:162
2710698992820,4cyclictest0-21swapper/210:55:262
2710698992820,4cyclictest0-21swapper/209:05:242
2710698992820,4cyclictest0-21swapper/209:05:242
271069899281,4cyclictest131rcu_preempt07:35:132
2710692992821,4cyclictest0-21swapper/110:45:251
2710692992821,4cyclictest0-21swapper/108:35:151
2710692992820,5cyclictest0-21swapper/109:50:181
2710692992820,4cyclictest0-21swapper/107:25:241
2710692992817,8cyclictest2712618-21idleruntime-cro07:15:011
2710688992823,2cyclictest13650irq/42-30be0000.ethernet08:21:530
2710688992811,3cyclictest0-21swapper/012:30:480
271068899280,14cyclictest0-21swapper/012:08:270
271069899274,4cyclictest131rcu_preempt11:05:112
271069899274,12cyclictest0-21swapper/208:35:252
271069899273,12cyclictest32-21ksoftirqd/209:59:242
271069899273,12cyclictest32-21ksoftirqd/209:59:242
2710698992719,5cyclictest0-21swapper/208:45:172
2710698992713,6cyclictest2781552-21kworker/u8:1+events_unbound11:23:492
2710692992719,5cyclictest0-21swapper/110:10:171
2710692992719,4cyclictest0-21swapper/109:00:151
2710692992718,4cyclictest2803479-21grep11:35:251
2710692992717,7cyclictest2752355-21sh09:10:001
2710692992717,7cyclictest2752355-21sh09:10:001
2710692992717,6cyclictest0-21swapper/109:20:261
2710692992714,8cyclictest215-21dbus-daemon11:20:031
2710698992620,4cyclictest2730124-21kworker/u8:1+events_unbound08:15:262
2710698992620,3cyclictest0-21swapper/209:45:252
2710698992619,5cyclictest2767537-21kworker/u8:2+events_unbound10:37:512
2710698992618,4cyclictest0-21swapper/210:05:142
2710698992618,4cyclictest0-21swapper/209:25:192
2710698992615,9cyclictest2710680-21cyclictest12:25:132
2710698992613,4cyclictest131rcu_preempt07:15:582
2710692992619,4cyclictest0-21swapper/109:20:011
2710692992618,4cyclictest0-21swapper/111:55:001
2710692992616,6cyclictest0-21swapper/109:35:001
2710692992615,6cyclictest2772616-21grep10:05:241
2710692992613,9cyclictest0-21swapper/107:50:161
2710698992516,7cyclictest2767537-21kworker/u8:2+events_unbound10:21:562
2710698992516,6cyclictest2817936-21wc12:20:132
271069899251,16cyclictest2802150-21kworker/u8:1+events_power_efficient11:40:112
271069899251,14cyclictest0-21swapper/208:01:052
2710692992519,3cyclictest0-21swapper/110:00:151
2710692992519,3cyclictest0-21swapper/109:40:191
2710692992519,3cyclictest0-21swapper/108:55:221
2710692992518,5cyclictest231ktimers/108:30:141
2710692992518,3cyclictest0-21swapper/110:25:161
2710688992519,3cyclictest0-21swapper/012:15:000
2710688992519,3cyclictest0-21swapper/007:45:000
2710698992418,4cyclictest2767537-21kworker/u8:2+events_unbound10:48:582
2710698992417,5cyclictest2750524-21kworker/u8:2+events_unbound09:32:292
271069899241,13cyclictest0-21swapper/208:33:402
2710688992414,5cyclictest2731459-21grep08:05:270
271069899236,10cyclictest0-21swapper/207:30:192
2710698992319,2cyclictest0-21swapper/211:55:142
2710698992319,2cyclictest0-21swapper/211:55:132
2710698992318,3cyclictest2728360-21kworker/u8:0-events_unbound08:05:262
2710698992317,4cyclictest2803899-21kworker/u8:0+events_unbound11:47:422
2710698992317,4cyclictest2730124-21kworker/u8:1+events_unbound08:14:542
271069299232,16cyclictest2810756-21cron12:00:001
271069299232,16cyclictest2810756-21cron12:00:001
2710692992317,3cyclictest0-21swapper/108:25:151
2710692992316,4cyclictest2769630-21sendmail10:00:001
2710692992316,4cyclictest2769630-21sendmail10:00:001
2710692992316,3cyclictest0-21swapper/107:40:161
271069299231,19cyclictest2728900-21cut08:00:171
2710692992311,7cyclictest2788376-21sh10:55:001
2710698992216,4cyclictest2743659-21kworker/u8:1+events_unbound09:02:062
2710692992216,3cyclictest0-21swapper/112:10:261
2710692992216,3cyclictest0-21swapper/108:55:011
2710692992214,4cyclictest2793897-21fschecks_count11:10:151
2710688992213,6cyclictest2780704-21awk10:30:250
2710692992113,5cyclictest215-21dbus-daemon08:09:271
2710692992113,5cyclictest0-21swapper/112:30:151
2710692991913,3cyclictest0-21swapper/107:30:241
2710692991911,5cyclictest231ktimers/107:45:141
2710692991911,5cyclictest231ktimers/107:45:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional