You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-24 - 17:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Mon Aug 24, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2517867995752,4cyclictest2772969-21gpgv11:15:003
2517862995042,5cyclictest2773402-21cp11:15:021
2517865994540,3cyclictest2747233-21apt-get10:50:102
2517865994438,4cyclictest2518082-21apt-get07:10:122
2517858994338,3cyclictest2673742-21cp09:40:010
2517867994238,3cyclictest2507623-21kworker/u8:1+flush-259:007:55:243
2517862994136,3cyclictest2731756-21apt-get10:35:111
2517862994033,5cyclictest2548767-21latency_hist07:40:011
2517858993935,3cyclictest2605487-21kworker/u8:2+flush-259:008:55:010
2517865993832,4cyclictest2850874-21latency_hist12:30:002
2517862993731,4cyclictest335557-21systemd-journal07:29:461
2517865993632,3cyclictest2564266-21kworker/u8:3+flush-259:009:05:232
2517862993630,4cyclictest2590285-21apt-get08:20:111
2517865993529,4cyclictest2559340-21apt-get07:50:092
2517867993431,2cyclictest2689576-21kworker/u8:2+flush-259:011:20:243
2517867993431,2cyclictest2625986-21kworker/u8:0+events_unbound09:15:143
2517858993429,3cyclictest2647252-21latency_hist09:15:010
2517867993330,2cyclictest2533019-21kworker/u8:0+flush-259:007:30:233
2517865993329,3cyclictest2564266-21kworker/u8:3+flush-259:010:40:242
2517858993329,3cyclictest2674068-21base6409:40:010
2517867993128,2cyclictest2850897-21gpgv12:30:003
2517865993126,3cyclictest2564266-21kworker/u8:3+events_unbound09:00:012
2517858993118,12cyclictest2787809-21ntpq11:25:210
2517867993026,3cyclictest2507623-21kworker/u8:1+flush-259:008:50:263
2517865992925,3cyclictest2625986-21kworker/u8:0+flush-259:009:30:132
2517858992924,4cyclictest335557-21systemd-journal11:59:090
2517858992824,3cyclictest2773544-21apt-get11:15:120
2517858992813,13cyclictest2614954-21ntpq08:40:200
2517867992723,3cyclictest2752377-21kworker/u8:4+events_unbound11:05:233
2517867992722,3cyclictest2778159-21kworker/u8:0+flush-259:011:50:173
2517858992724,2cyclictest2625986-21kworker/u8:0+flush-259:009:15:160
2517858992723,3cyclictest2600621-21apt-get08:30:120
2517867992623,2cyclictest2564266-21kworker/u8:3+flush-259:010:45:243
2517867992622,2cyclictest2778159-21kworker/u8:0+flush-259:012:10:193
2517867992619,5cyclictest335557-21systemd-journal10:15:003
2517867992619,5cyclictest2809455-21latency_hist11:50:013
2517865992621,4cyclictest2799335-21apt-get11:40:122
2517862992623,2cyclictest2721360-21apt-get10:25:121
2517862992620,4cyclictest2626322-21cp08:55:011
2517867992522,2cyclictest2710540-21kworker/u8:1+events_unbound10:15:173
2517867992522,2cyclictest2574207-21kworker/u8:2+events_unbound08:07:093
2517867992521,3cyclictest2564266-21kworker/u8:3+flush-259:008:19:593
2517867992521,3cyclictest2507623-21kworker/u8:1+flush-259:007:49:543
2517867992519,4cyclictest335557-21systemd-journal11:34:093
2517867992421,2cyclictest2746826-21kworker/u8:1+events_unbound10:50:253
2517867992421,2cyclictest2679245-21kworker/u8:0+flush-259:009:54:593
2517867992420,3cyclictest2778159-21kworker/u8:0+flush-259:012:05:113
2517867992420,3cyclictest2533019-21kworker/u8:0+events_unbound07:37:113
2517865992421,2cyclictest2625986-21kworker/u8:0+flush-259:009:10:232
2517865992420,3cyclictest2564266-21kworker/u8:3+flush-259:009:25:252
2517865992418,4cyclictest2673553-21latency_hist09:40:012
2517865992417,5cyclictest2653283-21diskstats09:20:122
2517858992419,3cyclictest2658272-21apt-get09:25:120
2517858992418,4cyclictest2528551-21diskstats07:20:150
2517867992320,2cyclictest2783603-21kworker/u8:1+events_unbound12:15:113
2517867992320,2cyclictest2564266-21kworker/u8:3+flush-259:008:30:253
2517867992319,3cyclictest3686945-21kworker/u8:2+events_unbound07:21:093
2517867992319,3cyclictest2830202-21kworker/u8:2+events_unbound12:30:163
2517867992319,3cyclictest2799073-21latency_hist11:40:013
2517867992319,3cyclictest2605487-21kworker/u8:2+events_unbound08:40:133
2517867992319,3cyclictest2579687-21kworker/u8:0+events_unbound08:20:263
2517867992318,3cyclictest2533019-21kworker/u8:0+events_unbound07:25:103
2517865992319,3cyclictest2778159-21kworker/u8:0+flush-259:012:00:262
2517865992319,3cyclictest2507623-21kworker/u8:1+events_unbound08:50:172
2517865992318,4cyclictest2772857-21apt-get11:15:022
2517865992318,4cyclictest2752164-21kworker/u8:3+events_unbound11:30:202
2517865992317,5cyclictest2678319-21ntpq09:40:212
2517862992319,3cyclictest2840755-21apt-get12:20:101
2517858992311,10cyclictest0-21swapper/010:45:000
2517867992219,3cyclictest2752377-21kworker/u8:4+events_unbound11:15:233
2517867992218,3cyclictest2564266-21kworker/u8:3+flush-259:009:40:163
2517867992217,3cyclictest509-21rs:main3
2517867992216,4cyclictest2694757-21latency_hist10:00:003
2517867992216,4cyclictest2647251-21apt-get09:15:013
251786299226,6cyclictest2584242-21sed08:10:251
251786299226,6cyclictest2584242-21sed08:10:241
2517862992217,4cyclictest2663448-21apt-get09:30:121
2517862992217,3cyclictest335557-21systemd-journal08:20:001
2517867992118,2cyclictest2507623-21kworker/u8:1+events_unbound07:54:593
2517867992116,3cyclictest2668585-21kworker/u8:4+events_unbound09:35:133
2517867992116,3cyclictest2564266-21kworker/u8:3+flush-259:009:05:143
2517867992115,5cyclictest2610674-21latency_hist08:40:013
2517865992119,1cyclictest2679245-21kworker/u8:0+events_unbound09:50:152
2517865992118,2cyclictest2679245-21kworker/u8:0+events_unbound10:10:142
2517865992117,3cyclictest2804261-21kworker/u8:2+events_unbound11:45:132
251786299219,3cyclictest2744711-21cut10:45:181
251786299215,5cyclictest335480-21sshd08:37:081
251785899217,12cyclictest0-21swapper/010:10:000
2517858992118,2cyclictest2625986-21kworker/u8:0+flush-259:009:35:010
2517858992118,2cyclictest2595179-21sh08:24:590
251785899211,6cyclictest2830166-21idleruntime-cro12:09:590
251785899211,6cyclictest2564227-21latency_hist07:55:000
2517867992018,2cyclictest3686945-21kworker/u8:2+flush-259:007:15:113
2517867992017,2cyclictest2689576-21kworker/u8:2+events_unbound10:27:103
2517867992017,2cyclictest2679245-21kworker/u8:0+events_unbound10:32:113
2517867992017,2cyclictest2679245-21kworker/u8:0+events_unbound10:32:113
2517867992013,5cyclictest335557-21systemd-journal11:30:003
2517865992016,3cyclictest2689576-21kworker/u8:2+flush-259:011:20:162
2517865992014,4cyclictest2710550-21/usr/sbin/munin10:15:122
2517865992013,5cyclictest2636118-21ntpq09:00:222
251786299203,6cyclictest2850158-21sensors12:25:241
251785899201,5cyclictest2763208-21ps11:05:170
2517858992015,3cyclictest2679495-21apt-get09:45:120
2517858992014,5cyclictest2568433-21ntpq07:55:200
2517858992014,4cyclictest2752146-21latency_hist10:55:010
2517858992014,4cyclictest2542740-21ntpq07:30:180
2517867991916,2cyclictest2689576-21kworker/u8:2+events_unbound10:39:103
2517867991916,2cyclictest2564266-21kworker/u8:3+flush-259:010:45:003
2517867991915,3cyclictest2778159-21kworker/u8:0+events_unbound12:00:403
2517865991915,3cyclictest3686945-21kworker/u8:2+events_unbound07:22:002
2517865991913,4cyclictest335557-21systemd-journal08:27:092
2517865991912,4cyclictest2720213-21ntpq10:20:212
251785899198,10cyclictest0-21swapper/010:30:180
251785899198,10cyclictest0-21swapper/010:30:180
251785899196,1cyclictest0-21swapper/012:20:240
251785899196,1cyclictest0-21swapper/008:25:230
251785899195,2cyclictest0-21swapper/010:29:550
251785899191,5cyclictest2851090-21apt-key12:30:000
251785899191,3cyclictest2590001-21cron08:20:000
251785899190,5cyclictest2840492-21latency_hist12:19:590
251785899190,5cyclictest2761620-21runrttasks11:00:210
251785899190,1cyclictest0-21swapper/009:50:220
2517867991815,2cyclictest2631783-21kworker/u8:1+events_unbound09:00:153
2517865991815,2cyclictest2564266-21kworker/u8:3+flush-259:010:25:142
2517865991815,2cyclictest2507623-21kworker/u8:1+events_unbound07:25:182
2517865991814,3cyclictest2679245-21kworker/u8:0+events_unbound10:05:242
2517865991814,3cyclictest2605487-21kworker/u8:2+events_unbound08:40:162
2517865991814,3cyclictest2579687-21kworker/u8:0+events_unbound08:10:252
2517865991814,3cyclictest2579687-21kworker/u8:0+events_unbound08:10:242
251786299183,5cyclictest2679496-21dpkg09:45:111
2517862991815,2cyclictest2752377-21kworker/u8:4+flush-259:011:10:141
2517862991813,3cyclictest2616128-21apt-get08:45:131
251785899185,1cyclictest0-21swapper/011:40:010
251785899184,3cyclictest507-21in:imuxsock12:04:550
251785899184,3cyclictest2626832-21apt-get08:55:120
251785899183,3cyclictest2695024-21apt-get10:00:130
251785899183,3cyclictest2605959-21cut08:35:140
251785899181,4cyclictest2641123-21tail09:05:190
2517858991813,4cyclictest60350irq/132-enp0s29f1:rx-007:15:330
251785899180,5cyclictest2861600-21apt12:40:010
251785899180,5cyclictest2808584-21ntpq11:45:200
251785899180,5cyclictest2538468-21/usr/sbin/munin07:30:000
251785899180,4cyclictest2636949-21cron09:04:590
2517867991714,2cyclictest2625986-21kworker/u8:0+flush-259:009:25:173
2517867991713,3cyclictest2850514-21kworker/u8:0+events_unbound12:38:103
2517867991713,3cyclictest2778159-21kworker/u8:0+events_unbound11:55:143
2517867991713,3cyclictest2631783-21kworker/u8:1+events_unbound09:46:093
2517867991712,4cyclictest2507623-21kworker/u8:1+events_unbound07:14:423
251786599172,8cyclictest2757369-21munin-plugin-st11:00:002
2517865991714,2cyclictest2533019-21kworker/u8:0+flush-259:007:40:132
2517865991713,3cyclictest2783603-21kworker/u8:1+events_unbound11:35:232
2517865991713,3cyclictest2689576-21kworker/u8:2+flush-259:009:55:192
251786299176,8cyclictest0-21swapper/110:10:141
2517862991715,1cyclictest2814610-21cron11:55:001
2517862991714,2cyclictest2820472-21irqcore12:00:171
2517862991713,3cyclictest335557-21systemd-journal11:24:091
2517862991713,3cyclictest335557-21systemd-journal09:09:091
2517862991713,3cyclictest2549008-21/usr/sbin/munin07:40:181
2517862991712,3cyclictest2559064-21latency_hist07:50:001
2517862991711,5cyclictest2543623-21cron07:34:591
251785899171,3cyclictest2705401-21apt-get10:10:110
2517858991713,3cyclictest60350irq/132-enp0s29f1:rx-008:46:530
2517858991713,3cyclictest2742040-21apt-get10:45:110
251785899170,4cyclictest2574509-21latency_hist08:05:010
2517867991613,2cyclictest2752164-21kworker/u8:3+flush-259:011:40:203
2517867991613,2cyclictest2689576-21kworker/u8:2+flush-259:010:20:153
2517865991612,3cyclictest2689576-21kworker/u8:2+flush-259:011:00:172
2517865991612,3cyclictest2689576-21kworker/u8:2+events_unbound11:05:182
2517865991611,3cyclictest2621009-21latency_hist08:50:012
251786299168,6cyclictest2839941-21ls12:15:251
251786299168,6cyclictest2763212-21sed11:05:171
251786299168,6cyclictest2673859-21apt-config09:40:011
251786299167,7cyclictest2830241-21sed12:10:001
251786299165,2cyclictest2636970-21munin-run09:05:011
2517862991613,2cyclictest2579682-21true08:10:001
2517862991612,3cyclictest507-21in:imuxsock08:35:001
2517862991610,4cyclictest2528122-21latency_hist07:20:011
251785899163,3cyclictest2814074-21/usr/sbin/munin11:50:250
251785899161,5cyclictest2694764-21/usr/sbin/munin10:00:000
251785899161,5cyclictest2547982-21/usr/sbin/munin07:35:240
251785899161,3cyclictest2553948-21unin-run07:45:000
2517858991613,2cyclictest60350irq/132-enp0s29f1:rx-011:30:140
2517858991613,2cyclictest60350irq/132-enp0s29f1:rx-011:20:250
2517858991613,2cyclictest60350irq/132-enp0s29f1:rx-010:19:090
2517858991611,4cyclictest60350irq/132-enp0s29f1:rx-008:10:260
2517858991611,4cyclictest60350irq/132-enp0s29f1:rx-008:10:250
251785899160,4cyclictest2559074-21awk07:49:590
2517867991512,2cyclictest2564266-21kworker/u8:3+events_unbound08:56:083
2517867991512,2cyclictest2533019-21kworker/u8:0+events_unbound07:40:153
251786599158,5cyclictest2578778-21ntpq08:05:222
2517865991512,2cyclictest2830178-21idleruntime-cro12:09:592
2517865991512,2cyclictest0-21swapper/207:39:592
2517865991511,3cyclictest2783603-21kworker/u8:1+flush-259:012:15:192
251786299159,5cyclictest2731489-21idleruntime-cro10:35:001
251786299159,5cyclictest2731489-21idleruntime-cro10:35:001
251786299159,4cyclictest2674678-21irqcore09:40:171
251786299159,4cyclictest2642653-21fw_conntrack09:10:141
251786299158,5cyclictest2700200-21taskset10:05:131
251786299158,4cyclictest2737041-21tr10:40:141
251786299157,7cyclictest2522113-21taskset07:10:221
251786299157,6cyclictest2752872-21cut10:55:161
251786299157,6cyclictest2715640-21cat10:20:001
251786299157,6cyclictest2528634-21fschecks_time07:20:141
251786299157,5cyclictest2559329-21acpi07:50:091
251786299155,2cyclictest2809462-21munin-run11:50:001
2517858991513,1cyclictest60350irq/132-enp0s29f1:rx-011:43:060
2517858991513,1cyclictest60350irq/132-enp0s29f1:rx-008:07:210
2517858991512,2cyclictest60350irq/132-enp0s29f1:rx-010:55:140
2517867991411,2cyclictest2756945-21unixbench_singl10:55:253
2517867991410,3cyclictest2689576-21kworker/u8:2+events_unbound11:00:223
2517867991410,3cyclictest2533019-21kworker/u8:0+flush-259:008:00:263
2517867991410,2cyclictest0-21swapper/309:25:013
2517865991412,1cyclictest0-21swapper/207:20:002
251786599141,13cyclictest0-21swapper/212:00:002
2517865991410,3cyclictest2625986-21kworker/u8:0+events_unbound09:15:242
2517865991410,3cyclictest2579687-21kworker/u8:0+events_unbound08:15:222
2517865991410,3cyclictest2564266-21kworker/u8:3+events_unbound10:35:222
2517865991410,3cyclictest2533019-21kworker/u8:0+events_unbound07:45:222
251786299149,3cyclictest2819776-21cron12:00:011
251786299149,3cyclictest2747646-21cut10:50:151
251786299148,4cyclictest2856819-21expr12:35:131
251786299148,4cyclictest2793887-21cron11:35:001
251786299148,4cyclictest2686955-21tail09:50:191
251786299148,4cyclictest2658680-21irqcore09:25:161
251786299148,4cyclictest2599886-21unixbench_multi08:25:261
251786299147,5cyclictest2851383-21base6412:30:001
251786299147,5cyclictest2699392-21ls10:00:251
251786299147,5cyclictest2690050-21grep09:55:151
251786299147,4cyclictest2799481-21cut11:40:141
251786299146,6cyclictest2832320-21kthreadcore12:10:191
251786299146,6cyclictest2797467-21grep11:35:191
251786299146,6cyclictest2630183-21grep08:55:181
251786299146,6cyclictest2614318-21kthreadcore08:40:181
251786299146,5cyclictest2788932-21cat11:30:011
251786299144,2cyclictest2716334-21df_inode10:20:131
251786299144,2cyclictest2568545-21ntp_states07:55:221
2517862991410,3cyclictest2657530-21tail09:20:241
2517858991410,1cyclictest60350irq/132-enp0s29f1:rx-012:10:170
2517867991310,2cyclictest2704707-21unixbench_singl10:05:253
2517867991310,2cyclictest2625986-21kworker/u8:0+flush-259:009:30:263
251786599139,3cyclictest2726544-21kworker/u8:4+flush-259:010:30:212
251786599139,3cyclictest2726544-21kworker/u8:4+flush-259:010:30:212
251786599139,2cyclictest2689576-21kworker/u8:2+flush-259:010:05:002
2517865991311,1cyclictest2564266-21kworker/u8:3+events_unbound10:45:172
251786299139,2cyclictest2569719-21cpuspeed_turbos08:00:141
251786299137,3cyclictest2648093-21cut09:15:131
251786299136,6cyclictest2758252-21sed11:00:171
2517858991312,1cyclictest60350irq/132-enp0s29f1:rx-007:12:060
2517858991311,1cyclictest60350irq/132-enp0s29f1:rx-011:10:200
2517858991311,1cyclictest60350irq/132-enp0s29f1:rx-010:35:140
2517858991310,2cyclictest60350irq/132-enp0s29f1:rx-010:20:240
2517858991310,2cyclictest60350irq/132-enp0s29f1:rx-009:20:440
251786799129,2cyclictest2564266-21kworker/u8:3+flush-259:008:28:353
251786799128,3cyclictest2507623-21kworker/u8:1+events_unbound08:10:273
251786799128,3cyclictest2507623-21kworker/u8:1+events_unbound08:10:273
251786599129,2cyclictest2850514-21kworker/u8:0+flush-259:012:35:172
251786599129,2cyclictest2605914-21cut08:35:132
251786599129,2cyclictest2600687-21cpuspeed_turbos08:30:132
251786599129,2cyclictest0-21swapper/208:00:002
251786599127,3cyclictest2788882-21sed11:30:012
251786799118,2cyclictest2695089-21cpuspeed_turbos10:00:143
251786599118,2cyclictest2564266-21kworker/u8:3+flush-259:009:45:202
251786599107,1cyclictest0-21swapper/208:00:092
251786599106,3cyclictest335557-21systemd-journal12:30:022
251786599106,2cyclictest2595175-21cron08:25:002
251786599101,7cyclictest335557-21systemd-journal12:24:592
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional