You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-02 - 03:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Wed Sep 02, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2488549994239,2cyclictest2624716-21gpgv21:20:011
2488545993732,3cyclictest2702243-21apt-get22:35:110
2488545993631,3cyclictest2546373-21base6420:05:010
2488549993530,3cyclictest2806459-21apt-get00:15:121
2488549993326,5cyclictest2660716-21latency_hist21:55:011
2488551993127,3cyclictest2499022-21apt-get19:20:122
2488545993126,3cyclictest2660975-21apt-get21:55:130
2488545993125,4cyclictest2812009-21cp00:20:020
2488555992926,2cyclictest2728056-21kworker/u8:2+flush-259:023:20:013
2488545992924,3cyclictest2639757-21kworker/u8:1+flush-259:023:00:000
2488545992916,11cyclictest2743601-21apt-get23:15:120
2488545992822,4cyclictest335557-21systemd-journal00:30:000
2488555992723,3cyclictest2639757-21kworker/u8:1+flush-259:022:25:013
2488551992724,2cyclictest2509345-21kworker/u8:0+flush-259:019:30:262
2488555992622,2cyclictest2618972-21kworker/u8:2+flush-259:021:20:253
2488545992620,4cyclictest2545961-21gpgv20:05:010
2488551992518,5cyclictest2612886-21ntpq21:05:202
2488549992520,3cyclictest2671307-21apt-get22:05:121
2488549992519,4cyclictest2566840-21latency_hist20:25:001
2488545992519,4cyclictest2525243-21cat19:45:010
2488549992420,2cyclictest0-21swapper/120:20:001
2488555992320,2cyclictest2498508-21kworker/u8:1+flush-259:019:35:013
2488555992318,3cyclictest2759511-21kworker/u8:1+events_unbound23:55:003
2488551992320,2cyclictest335557-21systemd-journal21:03:322
2488551992320,2cyclictest2681166-21kworker/u8:0+flush-259:022:20:172
2488551992318,4cyclictest335557-21systemd-journal23:14:402
2488551992318,4cyclictest335557-21systemd-journal23:14:402
2488549992318,3cyclictest2582788-21kworker/u8:3+flush-259:020:50:011
2488545992316,5cyclictest2659792-21ntpq21:50:210
2488555992218,3cyclictest2473172-21kworker/u8:2+flush-259:019:25:183
2488549992217,3cyclictest2796148-21apt-get00:05:121
2488549992216,4cyclictest2737294-21ntpq23:05:201
2488545992217,3cyclictest2681620-21/usr/sbin/munin22:15:150
2488545992216,4cyclictest2598300-21latency_hist20:55:000
2488555992115,4cyclictest2546271-21cat20:05:013
2488551992117,3cyclictest2509345-21kworker/u8:0+events_unbound19:40:242
2488551992116,4cyclictest2530726-21diskstats19:50:142
2488551992115,4cyclictest2582770-21latency_hist20:40:002
2488545992110,3cyclictest60350irq/132-enp0s29f1:rx-022:20:140
2488555992015,4cyclictest2765069-21cat23:35:013
2488551992017,2cyclictest2738412-21kworker/u8:3+events_unbound23:45:182
2488551992017,1cyclictest0-21swapper/221:55:002
2488549992017,2cyclictest2753688-21cut23:25:001
2488549992015,3cyclictest2525291-21/usr/sbin/munin19:45:201
2488549992013,4cyclictest335557-21systemd-journal22:34:591
248854999201,17cyclictest2816954-21true00:25:011
2488551991915,3cyclictest507-21in:imuxsock20:35:002
2488551991915,3cyclictest2712566-21kworker/u8:2+flush-259:022:45:172
2488549991914,3cyclictest2738446-21apt-get23:10:121
2488549991914,3cyclictest2738446-21apt-get23:10:111
2488549991913,4cyclictest2655555-21latency_hist21:50:011
2488549991912,5cyclictest2748522-21latency_hist23:20:001
2488545991916,2cyclictest1-21systemd20:28:310
2488555991816,1cyclictest0-21swapper/320:30:003
2488555991814,3cyclictest2635173-21apt-get21:30:123
2488555991814,3cyclictest2561890-21kworker/u8:0+flush-259:020:45:003
2488551991815,2cyclictest2473172-21kworker/u8:2+flush-259:019:25:242
2488549991812,4cyclictest2666126-21/usr/sbin/munin22:00:151
2488545991813,3cyclictest2759544-21apt-get23:30:110
2488545991812,4cyclictest2805269-21ntpq00:10:210
2488555991715,1cyclictest509-21rs:main3
2488551991715,1cyclictest0-21swapper/220:20:002
2488551991714,2cyclictest2582788-21kworker/u8:3+flush-259:021:15:272
2488551991714,1cyclictest0-21swapper/200:20:002
2488551991711,4cyclictest2498754-21latency_hist19:20:012
2488549991713,3cyclictest2738412-21kworker/u8:3+flush-259:023:29:131
2488549991713,3cyclictest2525284-21kworker/u8:4+flush-259:019:55:001
2488549991711,4cyclictest335557-21systemd-journal22:36:481
2488545991714,2cyclictest60350irq/132-enp0s29f1:rx-020:10:270
2488545991714,2cyclictest2540578-21sh20:00:010
2488545991714,1cyclictest0-21swapper/019:30:000
2488545991713,3cyclictest2618972-21kworker/u8:2+flush-259:021:20:320
2488545991711,4cyclictest2518555-21ntpq19:35:210
2488545991711,2cyclictest60350irq/132-enp0s29f1:rx-021:06:470
2488555991612,3cyclictest2822278-21gpgv00:30:013
2488555991612,2cyclictest0-21swapper/320:55:003
2488551991613,2cyclictest2791988-21kthreadcore00:00:172
2488551991613,2cyclictest2728056-21kworker/u8:2+events_unbound23:25:002
2488551991613,2cyclictest2639757-21kworker/u8:1+events_unbound22:05:242
2488551991612,3cyclictest509-21rs:main2
2488551991612,3cyclictest335557-21systemd-journal20:05:012
2488549991613,1cyclictest0-21swapper/119:20:001
2488549991611,3cyclictest509-21rs:main1
2488549991611,3cyclictest2770300-21/usr/sbin/munin23:40:241
2488545991614,1cyclictest60350irq/132-enp0s29f1:rx-021:35:230
2488545991614,1cyclictest60350irq/132-enp0s29f1:rx-019:30:210
2488545991613,2cyclictest60350irq/132-enp0s29f1:rx-023:50:230
2488545991613,2cyclictest60350irq/132-enp0s29f1:rx-023:06:170
2488545991613,2cyclictest2780355-21cron23:50:000
2488545991612,3cyclictest2770325-21apt-get23:40:110
2488545991612,3cyclictest2566842-21apt-get20:25:000
2488545991611,4cyclictest2498508-21kworker/u8:1+flush-259:020:15:340
2488545991610,4cyclictest2826861-21ntpq00:30:220
2488545991610,4cyclictest2572657-21/usr/sbin/munin20:30:160
2488545991610,4cyclictest2519897-21diskstats19:40:130
2488555991512,2cyclictest335557-21systemd-journal19:45:033
2488555991512,2cyclictest2759511-21kworker/u8:1+flush-259:000:05:223
2488555991512,2cyclictest2699395-21kthreadcore22:30:193
2488555991512,2cyclictest2686546-21cron22:20:003
2488555991511,3cyclictest2656018-21fschecks_count21:50:143
2488555991511,3cyclictest2639757-21kworker/u8:1+flush-259:022:13:033
248855199159,4cyclictest2732098-21ntpq23:00:182
2488551991513,1cyclictest2759280-21turbostat.cron23:30:002
2488551991512,2cyclictest2770046-21cron23:40:002
2488551991512,2cyclictest2498508-21kworker/u8:1+events_unbound19:55:272
248854999155,9cyclictest2727834-21awk22:59:591
2488549991511,3cyclictest2640031-21idleruntime-cro21:35:001
2488549991510,3cyclictest2541019-21cut20:00:141
2488545991514,1cyclictest60350irq/132-enp0s29f1:rx-021:28:020
2488545991513,1cyclictest60350irq/132-enp0s29f1:rx-023:23:040
2488545991513,1cyclictest60350irq/132-enp0s29f1:rx-023:10:200
2488545991513,1cyclictest60350irq/132-enp0s29f1:rx-023:10:190
2488545991513,1cyclictest60350irq/132-enp0s29f1:rx-022:40:230
2488545991513,1cyclictest60350irq/132-enp0s29f1:rx-022:10:210
2488545991513,1cyclictest60350irq/132-enp0s29f1:rx-021:30:190
2488545991513,1cyclictest60350irq/132-enp0s29f1:rx-019:20:250
2488545991512,2cyclictest60350irq/132-enp0s29f1:rx-021:00:000
2488545991511,3cyclictest2671067-21kworker/u8:3+flush-259:022:50:010
248854599150,14cyclictest1-21systemd20:38:310
2488555991412,1cyclictest0-21swapper/321:20:003
2488555991411,2cyclictest2737763-21unixbench_singl23:05:253
2488555991411,2cyclictest2639757-21kworker/u8:1+flush-259:021:35:233
2488555991411,1cyclictest0-21swapper/323:50:013
2488555991410,3cyclictest2567260-21gpgv20:25:003
248855199148,4cyclictest2696846-21latency_hist22:30:012
2488551991411,2cyclictest2561890-21kworker/u8:0+flush-259:020:45:262
2488551991410,3cyclictest2812284-21fschecks_time00:20:142
2488551991410,3cyclictest2764475-21sleep23:35:012
2488551991410,3cyclictest2618972-21kworker/u8:2+flush-259:021:30:242
248854999148,4cyclictest335557-21systemd-journal19:45:011
248854999147,5cyclictest335557-21systemd-journal22:29:591
248854999143,2cyclictest507-21in:imuxsock19:29:591
2488549991412,1cyclictest2573220-21irqcore20:30:181
2488549991411,2cyclictest2571436-21needreboot20:25:211
248854599149,4cyclictest2789857-21ntpq23:55:210
2488545991413,1cyclictest60350irq/132-enp0s29f1:rx-019:52:080
2488545991413,1cyclictest60350irq/132-enp0s29f1:rx-019:10:190
2488545991412,2cyclictest60350irq/132-enp0s29f1:rx-022:29:130
2488545991412,2cyclictest60350irq/132-enp0s29f1:rx-021:17:140
2488545991412,1cyclictest60350irq/132-enp0s29f1:rx-023:36:060
2488545991412,1cyclictest60350irq/132-enp0s29f1:rx-021:00:260
2488545991412,1cyclictest60350irq/132-enp0s29f1:rx-000:00:140
2488545991411,2cyclictest60350irq/132-enp0s29f1:rx-023:27:000
2488545991411,2cyclictest60350irq/132-enp0s29f1:rx-022:07:020
2488545991411,2cyclictest2811692-21latency_hist00:20:010
2488545991410,3cyclictest2796323-21fschecks_count00:05:150
248855599138,4cyclictest2109693-21snmpd21:40:163
248855599138,3cyclictest2706133-21grep22:35:193
2488555991310,2cyclictest2828484-21irqcore00:35:183
2488555991310,2cyclictest2805184-21needreboot00:10:213
2488555991310,2cyclictest2712566-21kworker/u8:2+flush-259:022:45:273
2488555991310,2cyclictest2671067-21kworker/u8:3+flush-259:022:55:233
2488555991310,2cyclictest2492632-21needreboot19:10:213
248855199139,3cyclictest2618972-21kworker/u8:2+events_unbound21:35:022
248855199131,4cyclictest2801033-21awk00:10:002
2488551991311,1cyclictest2519468-21cron19:40:002
2488551991311,1cyclictest0-21swapper/221:40:162
2488551991310,2cyclictest391rcuc/220:45:002
2488551991310,2cyclictest2825168-21tr00:30:182
2488551991310,2cyclictest2707123-21idleruntime-cro22:39:592
248854999139,3cyclictest2728056-21kworker/u8:2+flush-259:023:00:141
248854999139,3cyclictest2597858-21unixbench_singl20:50:241
248854999137,4cyclictest509-21rs:main1
248854999137,4cyclictest335557-21systemd-journal22:51:411
2488549991311,1cyclictest0-21swapper/120:40:171
2488549991310,2cyclictest0-21swapper/100:30:001
2488549991310,1cyclictest0-21swapper/119:30:191
2488545991311,1cyclictest60350irq/132-enp0s29f1:rx-023:00:230
2488545991311,1cyclictest60350irq/132-enp0s29f1:rx-021:13:570
2488545991311,1cyclictest60350irq/132-enp0s29f1:rx-020:43:000
2488545991311,1cyclictest60350irq/132-enp0s29f1:rx-019:18:340
2488545991311,1cyclictest60350irq/132-enp0s29f1:rx-000:35:210
2488545991310,2cyclictest60350irq/132-enp0s29f1:rx-020:45:160
248855599129,2cyclictest2561890-21kworker/u8:0+flush-259:020:35:203
248855599129,2cyclictest0-21swapper/320:35:003
248855599129,2cyclictest0-21swapper/319:45:013
248855599128,3cyclictest2618972-21kworker/u8:2+flush-259:021:25:053
248855599128,3cyclictest2598564-21apt-get20:55:123
248855599127,4cyclictest2557054-21fw_forwarded_lo20:15:153
2488555991210,1cyclictest2786222-21irqcore23:55:163
2488555991210,1cyclictest0-21swapper/300:20:003
248855199129,2cyclictest2618972-21kworker/u8:2+flush-259:021:20:202
248855199128,3cyclictest2525284-21kworker/u8:4+events_unbound19:45:182
248855199127,4cyclictest335557-21systemd-journal22:59:592
2488551991210,1cyclictest2565828-21needreboot20:20:202
2488551991210,1cyclictest0-21swapper/222:34:592
248854999129,2cyclictest2686087-21unixbench_multi22:15:261
248854999127,4cyclictest2639757-21kworker/u8:1+flush-259:021:40:161
248854999126,4cyclictest2681166-21kworker/u8:0+flush-259:022:40:161
248854999120,6cyclictest2665901-21idleruntime-cro22:00:011
248855599119,1cyclictest0-21swapper/319:25:003
248855599119,1cyclictest0-21swapper/300:30:133
248855599118,2cyclictest2681166-21kworker/u8:0+events_unbound22:25:193
248855599117,3cyclictest2774864-21vmstat23:40:253
248855599117,3cyclictest2604009-21fw_packets21:00:153
248855599117,3cyclictest2592901-21sshd20:47:313
248855599110,1cyclictest2743343-21awk23:15:003
248855599110,1cyclictest2743343-21awk23:15:003
248855199119,1cyclictest0-21swapper/222:44:592
248855199118,2cyclictest2634417-21unixbench_multi21:25:242
248855199115,4cyclictest335557-21systemd-journal00:00:012
248854999118,2cyclictest2603786-21cpuspeed_turbos21:00:131
248854999118,2cyclictest0-21swapper/121:05:161
248854999117,3cyclictest2618972-21kworker/u8:2+flush-259:021:36:461
248854999117,2cyclictest0-21swapper/119:25:001
248854999110,9cyclictest2691682-21cron22:25:011
248854999110,5cyclictest2717503-21idleruntime-cro22:49:591
248854599119,1cyclictest60350irq/132-enp0s29f1:rx-022:00:170
248854599118,2cyclictest60350irq/132-enp0s29f1:rx-022:30:150
2488545991110,1cyclictest60350irq/132-enp0s29f1:rx-021:47:490
248855599107,2cyclictest491ktimers/323:30:213
248855599107,2cyclictest2614276-21fschecks_count21:10:133
248855599107,2cyclictest0-21swapper/323:25:203
248855599107,2cyclictest0-21swapper/322:00:003
248855599107,1cyclictest0-21swapper/319:15:223
248855199108,1cyclictest0-21swapper/222:50:262
248855199107,2cyclictest2816783-21kworker/u8:2+flush-259:000:35:182
248854999106,3cyclictest2759511-21kworker/u8:1+flush-259:023:30:411
248854999101,8cyclictest0-21swapper/123:55:171
24885559996,2cyclictest2655098-21unixbench_multi21:45:273
24885559993,4cyclictest2722198-21grep22:50:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional