You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-08 - 23:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Thu Jan 08, 2026 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2499812994742,4cyclictest2620155-21dpkg09:05:513
2499805994238,3cyclictest2727367-21kworker/u8:4+flush-259:011:05:232
2499805994037,2cyclictest2620196-21kworker/u8:5+events_unbound09:45:202
2499805994037,2cyclictest2499796-21kworker/u8:1+events_unbound07:50:132
2499800994036,2cyclictest2631095-21kworker/u8:0+flush-259:009:30:191
2499799994035,4cyclictest252-21systemd-journal11:15:000
2499805993936,2cyclictest2727035-21kworker/u8:3+flush-259:011:20:212
2499805993834,3cyclictest2727035-21kworker/u8:3+events_unbound11:25:242
2499805993834,3cyclictest2620196-21kworker/u8:5+flush-259:009:35:192
2499800993834,3cyclictest2679771-21kworker/u8:1+flush-259:010:30:151
2499805993733,3cyclictest2499796-21kworker/u8:1+events_unbound07:15:142
2499805993632,3cyclictest2663665-21kworker/u8:0+flush-259:011:00:012
2499805993531,3cyclictest2440535-21kworker/u8:2+flush-259:008:15:242
2499800993530,3cyclictest2731965-21latency_hist11:05:011
2499800993529,4cyclictest2710601-21latency_hist10:40:011
2499799993532,2cyclictest2752526-21kworker/u8:1+flush-259:012:20:130
2499805993431,2cyclictest2752526-21kworker/u8:1+events_unbound12:10:172
2499800993431,2cyclictest2620196-21kworker/u8:5+flush-259:009:15:231
2499799993431,2cyclictest2768224-21apt-get11:40:100
2499805993330,2cyclictest2727367-21kworker/u8:4+events_unbound11:10:192
2499805993330,2cyclictest2499796-21kworker/u8:1+flush-259:008:00:012
2499805993329,3cyclictest2685094-21kworker/u8:3+flush-259:010:25:192
2499805993328,4cyclictest252-21systemd-journal08:25:002
2499812993226,4cyclictest2784061-21/usr/sbin/munin11:55:143
2499805993229,2cyclictest2752526-21kworker/u8:1+events_unbound11:30:182
2499805993229,2cyclictest2568011-21kworker/u8:3+flush-259:008:25:152
2499805993228,3cyclictest2727367-21kworker/u8:4+events_unbound11:40:172
2499805993228,3cyclictest2727035-21kworker/u8:3+events_unbound11:00:132
2499805993228,3cyclictest2679771-21kworker/u8:1+events_unbound10:50:132
2499805993127,3cyclictest2663665-21kworker/u8:0+events_unbound10:20:202
2499805993127,2cyclictest2440535-21kworker/u8:2+flush-259:007:35:012
2499800993128,2cyclictest2568011-21kworker/u8:3+flush-259:009:12:501
2499800993125,5cyclictest2620533-21latency_hist09:10:001
2499805993027,2cyclictest2620196-21kworker/u8:5+events_unbound09:15:222
2499805993026,3cyclictest2440535-21kworker/u8:2+events_unbound07:40:002
2499800993026,3cyclictest2752526-21kworker/u8:1+events_unbound11:55:201
2499800993026,3cyclictest2499796-21kworker/u8:1+flush-259:007:30:171
2499812992923,4cyclictest2695141-21latency_hist10:25:013
2499812992922,5cyclictest252-21systemd-journal09:30:573
2499805992926,2cyclictest2727367-21kworker/u8:4+flush-259:012:35:142
2499805992926,2cyclictest2631095-21kworker/u8:0+events_unbound09:20:122
2499805992925,3cyclictest2568011-21kworker/u8:3+events_unbound08:45:192
2499805992924,3cyclictest2700541-21/usr/sbin/munin10:30:142
2499805992923,4cyclictest2646951-21apt-get09:35:002
2499800992923,4cyclictest2680151-21diskstats10:10:151
2499812992821,5cyclictest2640917-21ntpq09:25:203
2499805992824,3cyclictest2669470-21kworker/u8:2+events_unbound10:00:182
2499805992824,3cyclictest2499796-21kworker/u8:1+flush-259:007:25:162
2499805992824,3cyclictest2440535-21kworker/u8:2+events_unbound07:40:132
2499812992721,4cyclictest2515454-21/usr/sbin/munin07:25:143
2499812992720,5cyclictest2778211-21latency_hist11:50:003
2499812992715,10cyclictest2530896-21/usr/sbin/munin07:40:133
2499805992724,2cyclictest2752526-21kworker/u8:1+flush-259:012:30:182
2499805992723,3cyclictest2773143-21kworker/u8:2+events_unbound11:55:002
2499799992724,2cyclictest2568011-21kworker/u8:3+flush-259:008:45:150
2499812992619,4cyclictest2747388-21latency_hist11:20:023
2499812992618,6cyclictest2631095-21kworker/u8:0+flush-259:009:20:163
2499805992622,3cyclictest2679771-21kworker/u8:1+events_unbound10:50:012
2499805992622,3cyclictest2568011-21kworker/u8:3+events_unbound08:30:222
2499805992619,2cyclictest2499796-21kworker/u8:1+events_unbound08:10:172
2499800992622,3cyclictest2657585-21kworker/u8:1+flush-259:009:45:241
2499800992622,3cyclictest2573756-21gpgv08:25:001
2499800992619,5cyclictest2641823-21latency_hist09:30:001
2499799992620,4cyclictest2684862-21latency_hist10:15:020
2499805992521,3cyclictest2752526-21kworker/u8:1+events_unbound11:45:132
2499805992521,3cyclictest2620196-21kworker/u8:5+flush-259:009:40:192
2499805992521,3cyclictest2568011-21kworker/u8:3+flush-259:008:50:152
2499805992521,3cyclictest2499796-21kworker/u8:1+events_unbound08:00:242
2499805992518,3cyclictest2727035-21kworker/u8:3+events_unbound11:15:172
2499800992522,2cyclictest2752526-21kworker/u8:1+flush-259:012:30:231
2499800992522,2cyclictest2657585-21kworker/u8:1+events_unbound09:50:181
2499800992520,4cyclictest2636414-21gpgv09:25:001
2499800992518,5cyclictest2556509-21gpgv08:05:011
2499799992522,2cyclictest2679771-21kworker/u8:1+flush-259:010:30:230
2499799992522,2cyclictest2568011-21kworker/u8:3+flush-259:008:55:170
2499812992420,3cyclictest2685094-21kworker/u8:3+flush-259:010:30:003
2499812992416,5cyclictest2761948-21ntpq11:30:193
2499805992421,2cyclictest2631095-21kworker/u8:0+events_unbound09:25:182
2499805992421,2cyclictest2499796-21kworker/u8:1+events_unbound08:05:022
2499805992421,2cyclictest2440535-21kworker/u8:2+events_unbound07:49:532
2499800992421,2cyclictest2568011-21kworker/u8:3+events_unbound08:25:161
2499800992420,3cyclictest2568011-21kworker/u8:3+flush-259:008:45:181
2499799992421,2cyclictest2657585-21kworker/u8:1+events_unbound09:50:220
2499799992421,2cyclictest2499796-21kworker/u8:1+flush-259:008:35:210
2499799992421,2cyclictest2440535-21kworker/u8:2+flush-259:008:15:160
2499799992420,3cyclictest2557457-21apt-get08:10:000
2499812992316,5cyclictest2625862-21latency_hist09:15:003
2499800992320,2cyclictest2727367-21kworker/u8:4+events_unbound11:30:001
2499800992320,2cyclictest2440535-21kworker/u8:2+flush-259:007:20:201
2499800992319,3cyclictest2727367-21kworker/u8:4+events_unbound11:50:201
2499800992317,4cyclictest2647645-21apt-get09:35:111
2499812992219,2cyclictest2556420-21gpgv08:05:013
2499812992215,4cyclictest2747626-21/usr/sbin/munin11:20:233
2499805992218,3cyclictest2620196-21kworker/u8:5+flush-259:009:10:192
2499805992218,3cyclictest2499796-21kworker/u8:1+flush-259:008:35:102
2499800992218,3cyclictest2727367-21kworker/u8:4+events_unbound12:05:231
2499800992218,3cyclictest2685094-21kworker/u8:3+events_unbound10:25:281
2499800992218,3cyclictest2685094-21kworker/u8:3+events_unbound10:15:201
2499800992218,3cyclictest2440535-21kworker/u8:2+flush-259:007:10:161
2499799992219,2cyclictest2752526-21kworker/u8:1+events_unbound12:00:130
2499799992216,4cyclictest252-21systemd-journal08:45:000
2499812992115,4cyclictest2499796-21kworker/u8:1+flush-259:008:35:003
249980599210,5cyclictest2610180-21idleruntime-cro09:00:002
2499800992118,2cyclictest2727367-21kworker/u8:4+flush-259:011:10:161
2499800992117,3cyclictest2727367-21kworker/u8:4+flush-259:011:20:001
2499800992117,3cyclictest2669470-21kworker/u8:2+flush-259:010:00:151
2499799992118,2cyclictest2727367-21kworker/u8:4+flush-259:011:55:150
2499799992117,3cyclictest2657585-21kworker/u8:1+flush-259:009:45:200
2499799992115,4cyclictest2550347-21ntpq07:55:200
249981299208,2cyclictest2737437-21cpuspeed_turbos11:10:113
2499812992010,1cyclictest0-21swapper/308:35:103
2499805992016,3cyclictest2685094-21kworker/u8:3+flush-259:010:40:162
2499800992017,2cyclictest2685094-21kworker/u8:3+flush-259:010:40:221
2499800992016,3cyclictest2727367-21kworker/u8:4+events_unbound11:41:211
2499800992016,3cyclictest2727035-21kworker/u8:3+flush-259:011:25:011
2499799992017,2cyclictest2568011-21kworker/u8:3+events_unbound08:25:200
2499799992016,3cyclictest2727035-21kworker/u8:3+events_unbound11:25:210
2499799992016,3cyclictest2568011-21kworker/u8:3+flush-259:009:20:010
249981299199,1cyclictest0-21swapper/312:05:113
2499812991913,4cyclictest2525755-21/usr/sbin/munin07:35:133
2499805991915,3cyclictest2727367-21kworker/u8:4+events_unbound12:23:542
2499805991915,3cyclictest2685094-21kworker/u8:3+events_unbound10:15:132
2499805991915,3cyclictest2440535-21kworker/u8:2+events_unbound07:10:122
2499800991916,2cyclictest2499796-21kworker/u8:1+events_unbound08:10:191
2499800991915,3cyclictest2727367-21kworker/u8:4+events_unbound12:35:151
2499800991915,3cyclictest2663665-21kworker/u8:0+flush-259:010:45:201
2499800991915,3cyclictest2499796-21kworker/u8:1+events_unbound07:25:171
2499800991914,4cyclictest2551232-21latency_hist08:00:011
2499800991914,3cyclictest252-21systemd-journal11:09:531
2499800991912,5cyclictest2793239-21ntpq12:00:191
2499799991916,2cyclictest2727367-21kworker/u8:4+events_unbound12:25:200
2499799991916,2cyclictest2685094-21kworker/u8:3+flush-259:010:40:250
2499799991916,2cyclictest2679771-21kworker/u8:1+flush-259:010:35:230
2499799991916,2cyclictest2669470-21kworker/u8:2+events_unbound10:05:170
2499799991915,3cyclictest2440535-21kworker/u8:2+events_unbound07:20:230
2499799991914,4cyclictest2799293-21latency_hist12:10:010
249979999191,17cyclictest1-21systemd12:35:400
249979999190,17cyclictest1-21systemd11:09:540
2499812991815,2cyclictest2706127-21irqcore10:35:153
2499812991814,3cyclictest2669470-21kworker/u8:2+flush-259:010:05:273
2499812991813,3cyclictest2600133-21/usr/sbin/munin08:50:133
2499812991812,4cyclictest2814761-21turbostat12:25:003
2499812991811,5cyclictest2710840-21/usr/sbin/munin10:40:133
2499805991816,1cyclictest2752526-21kworker/u8:1+events_unbound12:00:182
2499805991815,2cyclictest2752526-21kworker/u8:1+events_unbound12:18:532
2499805991815,2cyclictest2752526-21kworker/u8:1+events_unbound11:37:532
2499805991815,2cyclictest2663665-21kworker/u8:0+flush-259:010:10:242
2499800991815,2cyclictest2752526-21kworker/u8:1+events_unbound11:45:151
2499800991815,2cyclictest2679771-21kworker/u8:1+events_unbound10:50:151
2499800991815,2cyclictest2568011-21kworker/u8:3+flush-259:008:40:161
2499800991815,2cyclictest2440535-21kworker/u8:2+events_unbound08:15:231
2499800991814,3cyclictest2663665-21kworker/u8:0+events_unbound10:20:251
2499800991814,3cyclictest2440535-21kworker/u8:2+flush-259:007:35:191
2499800991812,4cyclictest2809573-21latency_hist12:20:011
2499800991812,4cyclictest2583583-21ntpq08:30:221
2499799991816,2cyclictest2499796-21kworker/u8:1+events_unbound07:30:160
2499799991815,2cyclictest2715996-21kworker/u8:4+flush-259:010:45:130
2499799991815,2cyclictest2663665-21kworker/u8:0+flush-259:010:00:010
2499799991815,2cyclictest2663665-21kworker/u8:0+events_unbound10:20:000
2499799991815,2cyclictest2499796-21kworker/u8:1+flush-259:008:20:150
2499799991815,2cyclictest2499796-21kworker/u8:1+flush-259:007:25:210
2499799991814,3cyclictest2636810-21base6409:25:010
2499799991813,4cyclictest2530650-21latency_hist07:40:010
249981299179,5cyclictest2540089-21ntpq07:45:213
2499805991713,3cyclictest2727367-21kworker/u8:4+events_unbound12:09:552
2499805991713,3cyclictest2727367-21kworker/u8:4+events_unbound11:55:162
2499805991713,3cyclictest2669470-21kworker/u8:2+events_unbound10:05:152
2499805991713,3cyclictest2657585-21kworker/u8:1+flush-259:009:50:192
2499805991713,3cyclictest2568011-21kworker/u8:3+events_unbound09:00:272
2499800991714,2cyclictest2620196-21kworker/u8:5+events_unbound09:56:531
2499800991713,3cyclictest2556956-21gpgv08:05:011
2499799991714,2cyclictest2727367-21kworker/u8:4+flush-259:011:45:180
2499799991714,2cyclictest2727035-21kworker/u8:3+flush-259:011:15:220
2499799991714,2cyclictest2679771-21kworker/u8:1+events_unbound10:53:550
2499799991714,2cyclictest2620196-21kworker/u8:5+flush-259:009:10:150
2499799991713,3cyclictest2752526-21kworker/u8:1+flush-259:011:30:140
2499799991713,3cyclictest2727367-21kworker/u8:4+events_unbound11:50:160
2499812991612,3cyclictest2663665-21kworker/u8:0+flush-259:010:46:533
2499812991611,3cyclictest2499796-21kworker/u8:1+flush-259:007:55:013
2499805991611,4cyclictest2568011-21kworker/u8:3+events_unbound09:05:122
2499800991613,2cyclictest2752526-21kworker/u8:1+events_unbound11:31:541
2499800991613,2cyclictest2727367-21kworker/u8:4+flush-259:011:35:201
2499800991613,2cyclictest2499796-21kworker/u8:1+events_unbound08:55:241
2499800991613,2cyclictest2499796-21kworker/u8:1+events_unbound08:35:151
2499800991612,3cyclictest2679771-21kworker/u8:1+events_unbound10:55:541
2499800991612,3cyclictest2657585-21kworker/u8:1+events_unbound09:40:541
2499800991612,3cyclictest2499796-21kworker/u8:1+events_unbound07:45:131
2499800991612,3cyclictest2440535-21kworker/u8:2+flush-259:007:40:231
2499799991613,2cyclictest2631095-21kworker/u8:0+events_unbound09:30:150
2499799991612,3cyclictest2727367-21kworker/u8:4+events_unbound12:15:540
2499799991612,3cyclictest2568011-21kworker/u8:3+events_unbound09:05:210
2499812991512,1cyclictest0-21swapper/308:10:003
2499812991511,3cyclictest507-21in:imuxsock07:34:593
2499812991511,2cyclictest0-21swapper/308:25:003
2499812991510,4cyclictest2558118-21/usr/sbin/munin08:10:103
2499805991512,2cyclictest2727367-21kworker/u8:4+events_unbound12:25:132
2499805991512,2cyclictest2709742-21ntp_kernel_pll_10:35:202
2499805991511,3cyclictest2499796-21kworker/u8:1+events_unbound07:20:152
2499800991512,2cyclictest2568011-21kworker/u8:3+flush-259:009:00:151
2499800991511,3cyclictest2669470-21kworker/u8:2+events_unbound10:09:541
2499799991513,1cyclictest60350irq/132-enp0s29f1:rx-011:35:170
2499799991513,1cyclictest60350irq/132-enp0s29f1:rx-011:20:240
2499799991513,1cyclictest60350irq/132-enp0s29f1:rx-010:25:170
2499799991513,1cyclictest60350irq/132-enp0s29f1:rx-010:20:130
2499799991513,1cyclictest60350irq/132-enp0s29f1:rx-010:00:190
2499799991513,1cyclictest60350irq/132-enp0s29f1:rx-009:42:360
2499799991513,1cyclictest60350irq/132-enp0s29f1:rx-007:40:150
2499799991513,1cyclictest2731966-21turbostat11:05:000
2499799991513,1cyclictest2440535-21kworker/u8:2+events_unbound07:10:210
2499799991512,2cyclictest60350irq/132-enp0s29f1:rx-012:10:180
2499799991512,2cyclictest2620196-21kworker/u8:5+events_unbound09:20:560
2499799991512,2cyclictest2440535-21kworker/u8:2+flush-259:007:45:210
249981299149,3cyclictest2767953-21dump-pmu-power11:39:593
249981299148,4cyclictest2757660-21latency_hist11:30:013
2499812991410,3cyclictest2653158-21/usr/sbin/munin09:40:143
2499812991410,2cyclictest0-21swapper/311:40:193
2499805991410,3cyclictest2620196-21kworker/u8:5+events_unbound09:55:132
2499800991410,3cyclictest2809837-21apt-get12:20:111
2499800991410,3cyclictest2752526-21kworker/u8:1+events_unbound12:11:251
249979999149,4cyclictest2620196-21kworker/u8:5+events_unbound09:38:530
2499799991413,1cyclictest60350irq/132-enp0s29f1:rx-007:50:250
2499799991412,1cyclictest60350irq/132-enp0s29f1:rx-012:30:410
2499799991412,1cyclictest60350irq/132-enp0s29f1:rx-008:11:420
2499799991411,2cyclictest60350irq/132-enp0s29f1:rx-007:15:120
2499799991411,2cyclictest2568011-21kworker/u8:3+events_unbound09:00:120
249981299139,3cyclictest491ktimers/312:15:113
249981299139,3cyclictest2679771-21kworker/u8:1+flush-259:010:30:193
249981299139,3cyclictest2668311-21unixbench_multi09:50:253
249981299137,4cyclictest2685094-21kworker/u8:3+flush-259:010:15:113
249981299136,5cyclictest2830225-21latency_hist12:40:013
249981299136,4cyclictest252-21systemd-journal09:40:013
2499812991310,2cyclictest2598915-21needreboot08:45:183
249980099139,3cyclictest2727367-21kworker/u8:4+events_unbound12:25:181
249979999139,3cyclictest2551571-21cpuspeed_turbos08:00:150
249979999139,3cyclictest2499796-21kworker/u8:1+events_unbound08:50:220
2499799991310,3cyclictest2679771-21kworker/u8:1+events_unbound10:58:530
249981299129,2cyclictest509-21rs:main3
249981299129,2cyclictest0-21swapper/312:25:173
249981299128,3cyclictest500-21systemd-logind11:05:013
2499812991210,1cyclictest0-21swapper/310:50:123
249980099128,3cyclictest2600158-21apt-get08:50:101
2499799991210,1cyclictest60350irq/132-enp0s29f1:rx-008:30:210
249981299117,3cyclictest0-21swapper/311:54:593
249981299116,4cyclictest252-21systemd-journal09:55:523
249981299111,7cyclictest2551232-21latency_hist08:00:013
249980599117,3cyclictest2499796-21kworker/u8:1+flush-259:008:40:412
249980099116,3cyclictest2505200-21apt-get07:15:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional