You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-29 - 13:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Wed Apr 29, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2520611995349,3cyclictest2542042-21kworker/u8:3+events_unbound19:35:220
2520623994943,4cyclictest2724324-21latency_hist22:24:593
2520611994945,3cyclictest2802294-21kworker/u8:2+events_unbound23:45:220
2520623994846,1cyclictest2541998-21lspci19:26:413
2520611994744,3cyclictest2536057-21kworker/u8:2+events_unbound19:26:590
2520611994739,2cyclictest2652188-21kworker/u8:3+events_unbound22:40:170
2520623994421,16cyclictest2842690-21ntpq00:15:213
2520623994236,4cyclictest2786973-21latency_hist23:25:013
2520614994235,5cyclictest2657639-21apt-get21:20:121
2520611994239,2cyclictest2693654-21kworker/u8:0+events_unbound22:05:140
2520618994138,2cyclictest2652188-21kworker/u8:3+flush-259:022:40:242
2520618994135,4cyclictest2837536-21ntpq00:10:212
2520611994138,2cyclictest2548055-21kworker/u8:1+events_unbound20:45:190
2520618994037,2cyclictest2461936-21kworker/u8:0+flush-259:019:45:152
2520611994037,2cyclictest2749453-21kworker/u8:0+events_unbound23:00:210
2520611994037,2cyclictest2611209-21kworker/u8:2+events_unbound21:15:230
2520611994037,2cyclictest2461936-21kworker/u8:0+events_unbound19:50:140
2520611993936,2cyclictest2812702-21kworker/u8:1+events_unbound23:55:230
2520611993934,4cyclictest2606081-21apt-get20:30:120
2520611993934,4cyclictest2593790-21ntpq20:15:200
2520611993835,2cyclictest2626465-21kworker/u8:0+events_unbound20:50:220
2520611993831,5cyclictest2828125-21latency_hist00:05:020
2520611993734,2cyclictest2755248-21kworker/u8:1+events_unbound22:55:220
2520611993732,3cyclictest2461936-21kworker/u8:0+events_unbound19:20:150
2520611993731,5cyclictest2801498-21ntpq23:35:210
2520623993631,3cyclictest2647030-21latency_hist21:10:013
2520618993629,5cyclictest252-21systemd-journal20:55:012
2520614993631,3cyclictest2756316-21apt-get22:55:121
2520614993629,5cyclictest2671949-21ntpq21:30:231
2520611993634,1cyclictest2776383-21kworker/u8:1+events_unbound23:15:160
2520611993632,3cyclictest2760956-21kworker/u8:3+events_unbound23:30:230
2520611993632,3cyclictest2652188-21kworker/u8:3+events_unbound22:25:240
2520611993632,3cyclictest2652188-21kworker/u8:3+events_unbound22:00:220
2520618993529,5cyclictest2542017-21getprofile19:26:412
2520614993531,3cyclictest2749453-21kworker/u8:0+flush-259:023:00:241
2520611993533,2cyclictest2548055-21kworker/u8:1+events_unbound21:05:240
2520611993531,3cyclictest2611209-21kworker/u8:2+flush-259:021:35:240
2520611993531,3cyclictest2611209-21kworker/u8:2+events_unbound21:25:200
2520611993531,3cyclictest2548055-21kworker/u8:1+events_unbound20:25:230
2520623993427,4cyclictest2708859-21latency_hist22:10:003
2520618993428,4cyclictest2755624-21latency_hist22:55:012
2520611993430,3cyclictest2548055-21kworker/u8:1+events_unbound20:35:170
2520611993330,2cyclictest2760956-21kworker/u8:3+events_unbound23:20:200
2520611993329,3cyclictest2626465-21kworker/u8:0+events_unbound21:00:160
2520611993329,3cyclictest2611209-21kworker/u8:2+flush-259:020:40:280
2520611993229,2cyclictest2724352-21kworker/u8:2+events_unbound23:05:170
2520611993228,3cyclictest2812702-21kworker/u8:1+flush-259:000:10:260
2520611993228,3cyclictest2678008-21kworker/u8:1+events_unbound21:58:530
2520611993228,3cyclictest2461936-21kworker/u8:0+events_unbound19:45:290
2520623993127,2cyclictest2652188-21kworker/u8:3+flush-259:022:00:163
2520618993128,2cyclictest2548055-21kworker/u8:1+flush-259:021:15:262
2520618993127,3cyclictest2678008-21kworker/u8:1+flush-259:021:55:252
2520618993124,4cyclictest252-21systemd-journal20:15:512
2520614993127,3cyclictest2812702-21kworker/u8:1+flush-259:000:25:241
2520611993128,2cyclictest2792138-21kworker/u8:0+events_unbound23:40:180
2520611993128,2cyclictest2652188-21kworker/u8:3+events_unbound22:50:200
2520611993127,3cyclictest2822964-21kworker/u8:3+events_unbound00:20:010
2520623993025,4cyclictest2776892-21apt-get23:15:123
2520618993026,3cyclictest2792138-21kworker/u8:0+events_unbound23:40:242
2520618993026,3cyclictest2760956-21kworker/u8:3+events_unbound23:30:012
2520611993027,2cyclictest2542042-21kworker/u8:3+events_unbound19:40:200
2520611993026,3cyclictest2652188-21kworker/u8:3+events_unbound22:10:210
2520611993026,3cyclictest2626465-21kworker/u8:0+flush-259:020:55:210
2520611993026,3cyclictest2461936-21kworker/u8:0+flush-259:020:10:510
2520618992925,3cyclictest2802294-21kworker/u8:2+flush-259:000:00:132
2520618992925,2cyclictest2542042-21kworker/u8:3+flush-259:019:35:272
2520614992925,3cyclictest2812702-21kworker/u8:1+flush-259:000:05:241
2520614992922,5cyclictest2604899-21ntpq20:25:201
2520611992927,2cyclictest2812702-21kworker/u8:1+events_unbound00:30:020
2520611992926,2cyclictest2611209-21kworker/u8:2+flush-259:021:30:150
2520611992926,2cyclictest2548055-21kworker/u8:1+events_unbound20:00:190
2520611992925,3cyclictest2812702-21kworker/u8:1+flush-259:000:25:250
2520611992921,6cyclictest2775768-21ntpq23:10:230
2520618992824,3cyclictest2461936-21kworker/u8:0+events_unbound20:10:222
2520614992824,3cyclictest2678008-21kworker/u8:1+events_unbound21:55:221
2520614992821,5cyclictest2657371-21latency_hist21:20:001
2520611992823,3cyclictest509-21rs:main0
2520623992720,5cyclictest2776629-21latency_hist23:15:013
2520618992723,3cyclictest2749453-21kworker/u8:0+events_unbound23:00:192
2520618992723,3cyclictest2548055-21kworker/u8:1+flush-259:020:05:492
2520618992722,4cyclictest2729691-21gpgv22:30:002
2520614992721,4cyclictest2816909-21ntpq23:50:211
2520611992724,2cyclictest2678008-21kworker/u8:1+flush-259:021:45:110
2520611992723,3cyclictest2652188-21kworker/u8:3+events_unbound21:20:270
2520623992621,3cyclictest2818057-21/usr/sbin/munin23:55:143
2520623992621,3cyclictest2626465-21kworker/u8:0+flush-259:020:55:173
2520623992620,4cyclictest2678249-21apt-get21:40:133
2520618992622,3cyclictest2812702-21kworker/u8:1+events_unbound00:08:562
2520618992622,3cyclictest2714063-21kworker/u8:1+flush-259:022:20:542
2520618992622,3cyclictest2548055-21kworker/u8:1+flush-259:020:30:252
2520618992621,3cyclictest2760956-21kworker/u8:3+flush-259:023:15:242
2520614992622,3cyclictest2693654-21kworker/u8:0+events_unbound22:05:251
2520614992622,3cyclictest2548055-21kworker/u8:1+flush-259:019:55:251
2520614992622,3cyclictest2548055-21kworker/u8:1+flush-259:019:55:251
2520614992620,4cyclictest252-21systemd-journal20:44:511
2520611992622,3cyclictest2461936-21kworker/u8:0+events_unbound20:05:260
2520623992521,3cyclictest2589480-21kworker/u8:4+flush-259:020:25:003
2520623992517,5cyclictest2828549-21diskstats00:05:143
2520618992522,2cyclictest2822964-21kworker/u8:3+events_unbound00:22:552
2520618992522,2cyclictest2802294-21kworker/u8:2+events_unbound23:55:572
2520618992522,2cyclictest2520360-21kworker/u8:1+flush-259:019:10:252
2520618992521,3cyclictest2812702-21kworker/u8:1+events_unbound23:50:222
2520618992521,3cyclictest2652188-21kworker/u8:3+events_unbound22:30:262
2520618992521,3cyclictest2461936-21kworker/u8:0+events_unbound19:55:152
2520618992521,3cyclictest2461936-21kworker/u8:0+events_unbound19:55:152
2520618992520,3cyclictest2812702-21kworker/u8:1+events_unbound00:40:012
2520614992521,3cyclictest2558594-21apt-get19:45:111
2520614992520,3cyclictest2760956-21kworker/u8:3+flush-259:023:30:211
2520611992522,2cyclictest2461936-21kworker/u8:0+events_unbound19:30:190
2520618992421,2cyclictest2652188-21kworker/u8:3+events_unbound22:35:252
2520618992421,2cyclictest2652188-21kworker/u8:3+events_unbound22:35:252
2520618992421,2cyclictest2611209-21kworker/u8:2+flush-259:021:50:252
2520618992421,2cyclictest2548055-21kworker/u8:1+events_unbound20:00:202
2520618992420,3cyclictest2636764-21kworker/u8:3+events_unbound21:01:522
2520618992420,3cyclictest2589480-21kworker/u8:4+events_unbound20:29:512
2520618992420,2cyclictest2611209-21kworker/u8:2+flush-259:021:35:192
2520614992420,3cyclictest2724352-21kworker/u8:2+flush-259:022:51:001
2520614992419,3cyclictest252-21systemd-journal19:34:511
2520614992418,4cyclictest509-21rs:main1
2520611992421,2cyclictest2714063-21kworker/u8:1+events_unbound22:35:010
2520611992421,2cyclictest2714063-21kworker/u8:1+events_unbound22:35:010
2520611992421,2cyclictest2520360-21kworker/u8:1+events_unbound19:15:140
2520611992420,3cyclictest2714063-21kworker/u8:1+events_unbound22:20:160
252062399237,14cyclictest2652150-21cron21:15:003
2520623992318,4cyclictest2714063-21kworker/u8:1+flush-259:022:45:003
2520618992319,3cyclictest2755248-21kworker/u8:1+events_unbound22:56:022
2520618992319,3cyclictest2611209-21kworker/u8:2+events_unbound21:25:402
2520618992315,6cyclictest2713126-21ntpq22:10:212
2520614992319,3cyclictest2678008-21kworker/u8:1+flush-259:021:45:201
2520614992318,3cyclictest2595355-21/usr/sbin/munin20:20:151
2520611992320,2cyclictest2589480-21kworker/u8:4+events_unbound20:25:010
2520623992218,2cyclictest2657373-21idleruntime-cro21:20:003
2520618992218,3cyclictest2760956-21kworker/u8:3+events_unbound23:30:132
2520618992218,3cyclictest2724352-21kworker/u8:2+events_unbound23:10:152
2520618992218,3cyclictest2548055-21kworker/u8:1+flush-259:020:45:162
2520618992216,4cyclictest2535713-21ntpq19:20:212
2520614992218,3cyclictest2812702-21kworker/u8:1+events_unbound00:20:211
2520614992218,3cyclictest2760956-21kworker/u8:3+flush-259:023:25:001
2520614992218,3cyclictest2548055-21kworker/u8:1+events_unbound21:05:151
2520614992218,3cyclictest2548055-21kworker/u8:1+events_unbound20:35:161
2520614992218,3cyclictest2548055-21kworker/u8:1+events_unbound20:30:211
2520611992219,2cyclictest2520360-21kworker/u8:1+events_unbound19:10:220
2520611992218,3cyclictest2611209-21kworker/u8:2+events_unbound21:14:510
2520611992218,3cyclictest2548055-21kworker/u8:1+events_unbound19:57:520
2520611992218,3cyclictest2548055-21kworker/u8:1+events_unbound19:57:510
252062399218,11cyclictest0-21swapper/300:15:003
2520623992114,5cyclictest2796425-21ntpq23:30:223
2520618992117,3cyclictest2678008-21kworker/u8:1+events_unbound21:45:232
2520618992117,3cyclictest2652188-21kworker/u8:3+flush-259:021:20:212
2520618992115,4cyclictest2858090-21ntpq00:30:222
2520614992118,2cyclictest2822964-21kworker/u8:3+flush-259:000:15:181
2520614992117,3cyclictest2812702-21kworker/u8:1+flush-259:000:30:351
2520611992118,2cyclictest2822964-21kworker/u8:3+events_unbound00:23:200
2520623992016,2cyclictest0-21swapper/300:34:593
2520623992015,4cyclictest2683365-21apt-get21:45:113
2520623992014,4cyclictest2853003-21ntpq00:25:223
2520618992017,3cyclictest2792138-21kworker/u8:0+events_unbound23:46:122
2520618992016,3cyclictest2714063-21kworker/u8:1+events_unbound22:45:132
2520611992015,4cyclictest2693654-21kworker/u8:0+flush-259:022:15:180
2520623991916,2cyclictest2817817-21turbostat.cron23:55:003
252062399191,3cyclictest2563531-21awk19:50:003
2520623991913,4cyclictest252-21systemd-journal23:03:533
2520618991916,2cyclictest2611209-21kworker/u8:2+flush-259:021:30:152
2520618991915,3cyclictest2760956-21kworker/u8:3+flush-259:023:20:192
2520618991915,3cyclictest2724352-21kworker/u8:2+events_unbound23:05:272
2520618991915,3cyclictest2548055-21kworker/u8:1+events_unbound21:10:202
2520618991914,3cyclictest2520360-21kworker/u8:1+events_unbound19:15:222
2520614991916,2cyclictest2693654-21kworker/u8:0+events_unbound22:15:241
2520614991915,3cyclictest2548055-21kworker/u8:1+events_unbound20:05:221
2520614991914,4cyclictest2647291-21apt-get21:10:131
2520611991916,2cyclictest2822964-21kworker/u8:3+events_unbound00:05:150
252062399189,7cyclictest0-21swapper/321:05:013
252062399186,10cyclictest0-21swapper/320:40:143
2520623991814,3cyclictest2729455-21apt-get22:30:023
252062399181,3cyclictest2848726-21latency_hist00:25:013
252062399181,3cyclictest2828117-21apt00:05:003
2520623991813,3cyclictest2548055-21kworker/u8:1+flush-259:020:25:213
2520623991812,4cyclictest2615223-21ntpq20:35:213
2520618991815,3cyclictest2652188-21kworker/u8:3+events_unbound21:40:172
2520614991815,2cyclictest2461936-21kworker/u8:0+flush-259:020:10:241
2520614991814,3cyclictest2776383-21kworker/u8:1+flush-259:023:35:221
2520614991814,3cyclictest2652188-21kworker/u8:3+flush-259:022:25:151
2520614991814,3cyclictest2652188-21kworker/u8:3+events_unbound22:00:201
2520614991813,4cyclictest2859246-21/usr/sbin/munin00:35:141
2520614991812,5cyclictest252-21systemd-journal20:04:501
2520614991812,4cyclictest2520774-21cat19:10:001
2520611991815,2cyclictest2776383-21kworker/u8:1+flush-259:023:25:240
2520611991815,2cyclictest2724352-21kworker/u8:2+events_unbound22:49:210
252062399171,5cyclictest2750000-21unixbench_multi22:45:243
252062399171,3cyclictest2761215-21turbostat23:00:003
252062399171,3cyclictest2719179-21sh22:20:003
252062399171,3cyclictest2677982-21latency_hist21:40:003
252062399171,3cyclictest2626424-21awk20:50:013
252062399171,3cyclictest2573487-21grep19:55:253
252062399171,3cyclictest2573487-21grep19:55:243
252062399171,3cyclictest2558322-21apt19:44:593
252062399171,3cyclictest2548025-21sh19:35:003
252062399171,2cyclictest2667663-21sh21:30:003
2520623991712,4cyclictest2652188-21kworker/u8:3+flush-259:022:25:173
2520623991711,4cyclictest2735291-21/usr/sbin/munin22:35:203
2520623991711,4cyclictest2735291-21/usr/sbin/munin22:35:203
252062399170,6cyclictest2713173-21ntpq22:10:223
252062399170,5cyclictest2803087-21grep23:40:193
252062399170,4cyclictest2594891-21taskset20:20:003
2520618991714,2cyclictest2548055-21kworker/u8:1+events_unbound20:36:512
2520618991713,3cyclictest2812702-21kworker/u8:1+events_unbound00:16:172
2520618991713,3cyclictest2693654-21kworker/u8:0+events_unbound22:15:172
2520614991714,2cyclictest2714063-21kworker/u8:1+events_unbound22:20:181
2520614991714,1cyclictest0-21swapper/121:00:011
2520614991713,3cyclictest2971-21cron20:50:011
2520614991713,3cyclictest2724352-21kworker/u8:2+events_unbound23:05:211
2520614991713,3cyclictest2520360-21kworker/u8:1+flush-259:019:15:201
2520611991714,2cyclictest2611209-21kworker/u8:2+events_unbound21:40:030
252062399161,4cyclictest2812649-21apt23:50:003
252062399161,3cyclictest2698554-21apt22:00:003
252062399161,2cyclictest2771489-21latency_hist23:10:003
252062399161,2cyclictest2531057-21turbostat19:19:593
252062399161,2cyclictest0-21swapper/320:00:233
252062399160,5cyclictest2581399-21grep20:05:183
252062399160,4cyclictest2610489-21tail20:30:243
252062399160,3cyclictest2802406-21idleruntime-cro23:40:003
2520618991612,3cyclictest2589480-21kworker/u8:4+flush-259:020:22:552
2520618991611,3cyclictest2693654-21kworker/u8:0+events_unbound22:05:192
2520618991611,3cyclictest2652188-21kworker/u8:3+events_unbound22:00:122
2520618991611,3cyclictest2548055-21kworker/u8:1+flush-259:019:45:002
2520614991612,3cyclictest2611209-21kworker/u8:2+events_unbound21:25:181
2520614991612,3cyclictest2548055-21kworker/u8:1+flush-259:019:35:201
2520614991612,3cyclictest2461936-21kworker/u8:0+flush-259:019:50:151
2520614991612,3cyclictest2461936-21kworker/u8:0+events_unbound19:20:211
2520614991610,4cyclictest2744402-21ntpq22:40:221
2520611991614,1cyclictest60350irq/132-enp0s29f1:rx-021:50:260
2520611991613,2cyclictest60350irq/132-enp0s29f1:rx-023:50:150
252062399151,3cyclictest2584939-21irqcore20:10:173
252062399151,3cyclictest2548712-21sed19:35:183
252062399150,4cyclictest2860385-21grep00:35:183
252062399150,4cyclictest2750843-21grep22:50:143
252062399150,4cyclictest2568449-21time-dir19:51:593
252062399150,4cyclictest2532759-21tail19:20:173
252062399150,4cyclictest2522061-21ps19:10:183
252062399150,4cyclictest2520320-21latency_hist19:10:013
252062399150,3cyclictest2668049-21cpuspeed_turbos21:30:143
252062399150,2cyclictest2658060-21grep21:20:173
252061899151,3cyclictest2636750-21awk21:00:012
2520618991510,3cyclictest252-21systemd-journal19:50:202
252061499158,5cyclictest2692510-21ntpq21:50:221
252061499158,5cyclictest252-21systemd-journal00:15:001
2520614991512,2cyclictest2802294-21kworker/u8:2+flush-259:023:45:141
252062399140,4cyclictest2688993-21cut21:50:183
2520618991411,2cyclictest2548055-21kworker/u8:1+events_unbound20:42:502
2520618991410,3cyclictest2548055-21kworker/u8:1+events_unbound21:05:282
252061499149,4cyclictest2611209-21kworker/u8:2+flush-259:021:35:241
252061499148,4cyclictest252-21systemd-journal21:05:011
2520614991412,1cyclictest2594676-21cron20:20:001
2520614991410,3cyclictest2760956-21kworker/u8:3+events_unbound23:25:141
2520611991412,1cyclictest60350irq/132-enp0s29f1:rx-000:35:520
2520618991310,2cyclictest2812702-21kworker/u8:1+events_unbound00:25:152
2520618991310,2cyclictest2760956-21kworker/u8:3+flush-259:023:35:262
252061499137,4cyclictest252-21systemd-journal22:15:001
2520614991310,2cyclictest2739803-21unixbench_singl22:35:251
2520614991310,2cyclictest2739803-21unixbench_singl22:35:251
252061899129,2cyclictest2543129-21cpuspeed_turbos19:30:132
252061499129,2cyclictest0-21swapper/120:50:201
252061499127,4cyclictest2724352-21kworker/u8:2+events_unbound23:10:171
252061499119,1cyclictest0-21swapper/119:40:171
252061499118,2cyclictest2776383-21kworker/u8:1+events_unbound23:15:181
252061499107,1cyclictest0-21swapper/119:10:211
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional