You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-05 - 16:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sun Jul 05, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2525476995448,4cyclictest2841924-21latency_hist12:15:002
2525472995348,4cyclictest2466571-21kworker/u8:1+flush-259:007:45:011
2525472995047,2cyclictest2711863-21kworker/u8:0+flush-259:010:25:011
2525478994639,5cyclictest2706258-21latency_hist10:05:013
2525472994440,3cyclictest2628950-21kworker/u8:0+flush-259:009:30:161
2525472994239,2cyclictest2774739-21kworker/u8:4+events_unbound11:30:231
2525472994239,2cyclictest2628950-21kworker/u8:0+flush-259:009:39:591
2525478994135,4cyclictest2613251-21latency_hist08:35:003
2525472994138,2cyclictest2841943-21kworker/u8:1+events_unbound12:19:591
2525472994138,2cyclictest2800737-21kworker/u8:0+events_unbound12:05:231
2525476993933,4cyclictest2530553-21latency_hist07:15:002
2525472993936,2cyclictest2572205-21kworker/u8:3+events_unbound08:41:501
2525472993935,3cyclictest2800737-21kworker/u8:0+events_unbound12:00:201
2525472993931,2cyclictest2546623-21kworker/u8:2+events_unbound07:30:231
2525472993931,2cyclictest2546623-21kworker/u8:2+events_unbound07:30:221
2525472993835,2cyclictest2628950-21kworker/u8:0+events_unbound09:25:211
2525478993731,4cyclictest2546391-21latency_hist07:30:003
2525472993734,2cyclictest3652654-21kworker/u8:0+flush-259:007:15:221
2525472993734,2cyclictest2628950-21kworker/u8:0+events_unbound09:45:151
2525472993633,2cyclictest2628950-21kworker/u8:0+flush-259:009:10:191
2525472993633,2cyclictest2572205-21kworker/u8:3+events_unbound08:30:201
2525472993630,2cyclictest2764459-21kworker/u8:1+events_unbound11:25:201
2525468993632,3cyclictest2572205-21kworker/u8:3+flush-259:008:15:000
2525476993529,4cyclictest2759283-21/usr/sbin/munin10:55:142
2525468993531,3cyclictest2572205-21kworker/u8:3+flush-259:008:05:210
2525478993429,3cyclictest2665004-21cat09:25:013
2525472993431,2cyclictest2717239-21kworker/u8:1+flush-259:010:15:581
2525472993330,2cyclictest2743810-21kworker/u8:0+flush-259:011:00:271
2525472993329,3cyclictest2628950-21kworker/u8:0+flush-259:009:40:221
2525468993326,5cyclictest2835915-21ntpq12:05:200
2525472993229,2cyclictest2572205-21kworker/u8:3+flush-259:008:35:151
2525472993228,3cyclictest2831663-21kworker/u8:3+events_unbound12:30:151
2525472993227,4cyclictest2572183-21latency_hist07:55:001
2525472993225,3cyclictest2628950-21kworker/u8:0+events_unbound09:15:231
2525468993229,2cyclictest2800737-21kworker/u8:0+flush-259:011:35:260
2525478993124,5cyclictest2582461-21latency_hist08:05:013
2525476993125,4cyclictest2592681-21latency_hist08:15:012
2525472993128,2cyclictest2774739-21kworker/u8:4+events_unbound11:20:091
2525472993128,2cyclictest2669392-21kworker/u8:2+events_unbound10:10:191
2525472993128,2cyclictest2603193-21kworker/u8:4+flush-259:009:20:231
2525472993127,3cyclictest2826484-21latency_hist12:00:001
2525468993127,3cyclictest2603193-21kworker/u8:4+events_unbound09:00:260
2525468993127,2cyclictest2466571-21kworker/u8:1+flush-259:007:55:160
2525468993127,2cyclictest1-21systemd12:15:000
2525476993025,3cyclictest2659600-21latency_hist09:20:012
2525472993028,2cyclictest2466571-21kworker/u8:1+events_unbound08:24:491
2525472993027,2cyclictest2717239-21kworker/u8:1+events_unbound10:30:021
2525472993026,3cyclictest2837078-21apt-get12:10:141
2525472993026,3cyclictest2800737-21kworker/u8:0+flush-259:011:35:151
2525472993026,3cyclictest2546623-21kworker/u8:2+flush-259:007:45:211
2525472993025,3cyclictest2685772-21kworker/u8:3+events_unbound10:55:181
2525468993026,3cyclictest2717239-21kworker/u8:1+flush-259:010:40:180
2525468993026,3cyclictest2628950-21kworker/u8:0+flush-259:009:15:210
2525478992925,3cyclictest2546623-21kworker/u8:2+flush-259:007:49:563
2525478992924,4cyclictest335557-21systemd-journal09:29:593
2525478992922,5cyclictest2556744-21latency_hist07:40:003
2525476992925,3cyclictest2784878-21gpgv11:20:012
2525472992926,2cyclictest2743810-21kworker/u8:0+events_unbound10:50:201
2525472992926,2cyclictest2685772-21kworker/u8:3+flush-259:010:00:191
2525468992925,3cyclictest2852455-21kworker/u8:4+flush-259:012:30:210
2525478992822,4cyclictest2821323-21latency_hist11:55:003
2525472992825,2cyclictest2572205-21kworker/u8:3+flush-259:008:25:191
2525472992824,3cyclictest2841943-21kworker/u8:1+events_unbound12:20:101
2525472992824,3cyclictest2572205-21kworker/u8:3+events_unbound08:00:221
2525472992823,3cyclictest2628950-21kworker/u8:0+events_unbound09:05:221
2525468992825,2cyclictest2572205-21kworker/u8:3+flush-259:008:00:160
2525468992824,3cyclictest3652654-21kworker/u8:0+events_unbound07:30:000
2525468992824,3cyclictest2603193-21kworker/u8:4+events_unbound09:05:230
2525478992723,2cyclictest2738705-21apt-get10:35:123
2525478992722,4cyclictest2707108-21diskstats10:05:143
2525478992722,4cyclictest2707108-21diskstats10:05:133
2525478992722,4cyclictest2628950-21kworker/u8:0+flush-259:009:00:013
2525472992724,2cyclictest2805898-21kworker/u8:2+flush-259:011:40:231
2525472992724,2cyclictest2717239-21kworker/u8:1+events_unbound10:25:211
2525472992723,3cyclictest2685772-21kworker/u8:3+events_unbound11:10:131
2525472992723,2cyclictest2743810-21kworker/u8:0+flush-259:010:45:201
2525472992723,2cyclictest2685772-21kworker/u8:3+flush-259:009:55:191
2525468992723,3cyclictest2821546-21kworker/u8:2+events_unbound11:58:100
2525478992614,10cyclictest2572418-21/usr/sbin/munin07:55:183
2525476992622,3cyclictest2717271-21apt-get10:15:102
2525472992623,2cyclictest2800737-21kworker/u8:0+flush-259:012:35:261
2525472992623,2cyclictest2717239-21kworker/u8:1+events_unbound10:40:141
2525472992622,3cyclictest2852455-21kworker/u8:4+flush-259:012:25:251
2525472992622,3cyclictest2852455-21kworker/u8:4+flush-259:012:25:251
2525472992620,4cyclictest2541540-21apt-get07:25:111
2525468992623,2cyclictest2743810-21kworker/u8:0+events_unbound10:45:130
2525468992623,2cyclictest2628950-21kworker/u8:0+flush-259:008:55:160
2525468992622,3cyclictest2852455-21kworker/u8:4+events_unbound12:25:230
2525468992622,3cyclictest2852455-21kworker/u8:4+events_unbound12:25:230
2525468992622,3cyclictest2764459-21kworker/u8:1+events_unbound11:40:070
2525468992621,4cyclictest2733348-21cat10:30:010
2525468992619,5cyclictest335557-21systemd-journal08:40:010
2525476992521,3cyclictest2628950-21kworker/u8:0+flush-259:009:44:592
2525476992520,3cyclictest2806122-21/usr/sbin/munin11:40:182
2525472992521,3cyclictest2628950-21kworker/u8:0+events_unbound08:55:141
2525472992518,5cyclictest2743579-21latency_hist10:40:011
2525468992522,2cyclictest2695949-21kworker/u8:1+flush-259:010:05:240
2525468992522,2cyclictest2695949-21kworker/u8:1+flush-259:010:05:240
2525468992522,2cyclictest1-21systemd08:21:500
2525468992521,3cyclictest2717239-21kworker/u8:1+events_unbound10:35:220
2525478992420,3cyclictest2821589-21apt-get11:55:123
2525478992418,4cyclictest2790439-21latency_hist11:25:013
2525476992420,3cyclictest2717239-21kworker/u8:1+flush-259:010:35:162
2525472992421,2cyclictest2815888-21kworker/u8:3+events_unbound11:50:181
2525472992421,2cyclictest2466571-21kworker/u8:1+events_unbound08:05:221
2525468992421,2cyclictest2466571-21kworker/u8:1+events_unbound07:40:260
2525468992420,3cyclictest2774739-21kworker/u8:4+flush-259:011:25:010
2525468992420,3cyclictest2603193-21kworker/u8:4+flush-259:008:40:210
2525468992418,4cyclictest2717246-21/usr/sbin/munin10:15:130
2525478992317,5cyclictest335557-21systemd-journal09:10:003
2525476992319,3cyclictest2466571-21kworker/u8:1+flush-259:007:25:172
2525472992320,2cyclictest2695949-21kworker/u8:1+events_unbound10:05:161
2525472992320,2cyclictest2695949-21kworker/u8:1+events_unbound10:05:151
2525468992320,2cyclictest3652654-21kworker/u8:0+events_unbound07:10:140
2525468992320,2cyclictest2764459-21kworker/u8:1+events_unbound11:15:120
2525468992320,2cyclictest2628950-21kworker/u8:0+events_unbound08:50:190
2525468992319,3cyclictest2847315-21kworker/u8:2+flush-259:012:20:200
2525468992319,3cyclictest2743810-21kworker/u8:0+flush-259:011:10:160
2525468992318,4cyclictest2546623-21kworker/u8:2+flush-259:007:45:160
2525468992317,4cyclictest335557-21systemd-journal07:55:010
252546899230,21cyclictest1-21systemd07:30:510
252546899230,21cyclictest1-21systemd07:30:500
252547699227,4cyclictest2695037-21ntpq09:50:202
252547699226,4cyclictest2566179-21ld07:45:212
2525476992217,4cyclictest2603193-21kworker/u8:4+flush-259:009:25:002
2525472992219,2cyclictest2805898-21kworker/u8:2+events_unbound11:45:161
2525472992219,2cyclictest2572205-21kworker/u8:3+flush-259:008:15:191
2525472992218,3cyclictest2572205-21kworker/u8:3+events_unbound08:50:131
2525472992216,4cyclictest335557-21systemd-journal07:21:471
2525468992219,2cyclictest2774739-21kworker/u8:4+events_unbound11:30:150
2525468992219,2cyclictest2717239-21kworker/u8:1+flush-259:010:25:010
2525468992219,2cyclictest2603193-21kworker/u8:4+events_unbound08:46:510
2525468992218,3cyclictest2743810-21kworker/u8:0+flush-259:011:05:200
2525468992218,3cyclictest2669392-21kworker/u8:2+flush-259:009:40:240
2525468992218,2cyclictest0-21swapper/008:30:000
2525478992118,2cyclictest2764459-21kworker/u8:1+flush-259:011:15:143
2525478992118,2cyclictest2717239-21kworker/u8:1+flush-259:010:20:173
2525478992116,4cyclictest2674554-21ntpq09:30:203
2525476992118,2cyclictest2831663-21kworker/u8:3+flush-259:012:30:172
2525476992118,2cyclictest2466571-21kworker/u8:1+flush-259:007:55:222
2525472992117,3cyclictest2466571-21kworker/u8:1+events_unbound07:55:191
2525468992117,3cyclictest2831663-21kworker/u8:3+events_unbound12:15:200
2525468992117,3cyclictest2603193-21kworker/u8:4+flush-259:009:20:200
2525478992016,3cyclictest2535831-21gpgv07:20:003
2525478992015,4cyclictest2852230-21latency_hist12:24:593
2525478992015,4cyclictest2525674-21apt-get07:10:123
252547899201,4cyclictest2628730-21awk08:50:003
2525476992015,4cyclictest2536350-21cp07:20:012
2525472992017,2cyclictest2743810-21kworker/u8:0+events_unbound11:05:211
2525472992016,3cyclictest2669392-21kworker/u8:2+flush-259:009:50:191
2525472992016,3cyclictest2466571-21kworker/u8:1+events_unbound08:13:481
2525472992014,4cyclictest2466571-21kworker/u8:1+events_unbound07:38:521
2525468992016,3cyclictest2764459-21kworker/u8:1+events_unbound12:00:410
2525468992016,3cyclictest2685772-21kworker/u8:3+events_unbound11:00:230
2525468992016,3cyclictest2603193-21kworker/u8:4+events_unbound09:12:510
2525468992016,3cyclictest2466571-21kworker/u8:1+events_unbound07:20:270
2525478991917,1cyclictest2753888-21munin-plugin-st10:50:003
252547899191,2cyclictest2587587-21latency_hist08:10:003
252547699195,4cyclictest2731917-21ld10:25:212
2525476991915,3cyclictest2546623-21kworker/u8:2+flush-259:007:50:142
2525476991915,2cyclictest2774505-21sh11:10:012
2525472991915,3cyclictest2572205-21kworker/u8:3+events_unbound08:45:241
2525468991916,3cyclictest2685772-21kworker/u8:3+events_unbound10:55:020
2525468991916,2cyclictest2628950-21kworker/u8:0+events_unbound09:50:180
2525468991915,3cyclictest2800737-21kworker/u8:0+flush-259:011:49:260
252547899183,12cyclictest2759528-21cut10:55:153
2525478991815,2cyclictest335480-21sshd10:50:293
252547899181,15cyclictest2717004-21cron10:15:003
252547899181,14cyclictest2680611-21latency_hist09:40:013
252547899180,3cyclictest2847083-21latency_hist12:20:003
252547899180,2cyclictest2623556-21latency_hist08:45:023
2525476991815,2cyclictest2587574-21idleruntime-cro08:10:002
2525476991814,3cyclictest2628950-21kworker/u8:0+flush-259:009:35:162
2525476991814,3cyclictest2466571-21kworker/u8:1+flush-259:008:20:232
2525476991813,4cyclictest335557-21systemd-journal08:50:502
2525476991812,4cyclictest2649325-21latency_hist09:10:012
2525472991814,3cyclictest3652654-21kworker/u8:0+events_unbound07:11:431
2525468991814,3cyclictest2743810-21kworker/u8:0+events_unbound10:51:580
2525468991814,3cyclictest2695949-21kworker/u8:1+events_unbound10:04:550
2525468991814,3cyclictest2546623-21kworker/u8:2+events_unbound07:35:510
252546899180,16cyclictest1-21systemd09:56:570
252547899177,8cyclictest2732828-21latency_hist10:30:013
252547899172,3cyclictest2571337-21ntpq07:50:193
252547899171,6cyclictest2748743-21sh10:45:003
2525478991714,2cyclictest2598520-21irqcore08:20:153
252547899171,3cyclictest2722152-21apt10:19:593
2525478991713,3cyclictest2628950-21kworker/u8:0+flush-259:009:40:153
252547899171,2cyclictest2856563-21ntpq12:25:213
252547899171,2cyclictest2856563-21ntpq12:25:213
2525478991712,4cyclictest2764459-21kworker/u8:1+flush-259:011:25:233
252547899171,14cyclictest2618396-21cron08:40:003
2525478991710,6cyclictest0-21swapper/312:10:203
2525478991710,5cyclictest2774501-21latency_hist11:10:013
252547899170,6cyclictest2654457-21cron09:15:003
252547899170,4cyclictest2536196-21apt-config07:20:013
252547699176,3cyclictest2852659-21diskstats12:25:142
252547699176,3cyclictest2852659-21diskstats12:25:142
2525476991713,3cyclictest2800737-21kworker/u8:0+flush-259:012:35:162
2525472991714,2cyclictest2764459-21kworker/u8:1+events_unbound11:15:261
2525472991714,2cyclictest2603193-21kworker/u8:4+flush-259:009:00:221
2525468991710,5cyclictest2712249-21diskstats10:10:130
252547899164,9cyclictest0-21swapper/309:00:143
252547899163,11cyclictest2800699-21cron11:35:003
252547899161,4cyclictest335557-21systemd-journal08:15:013
2525478991614,1cyclictest2693389-21kthreadcore09:50:183
252547899161,3cyclictest2832175-21cut12:05:143
252547899161,3cyclictest2778781-21ntpq11:10:203
252547899161,2cyclictest2700242-21ntpq09:55:213
252547899161,2cyclictest2658743-21ntpq09:15:213
252547899161,2cyclictest2629407-21grep08:50:173
252547899161,2cyclictest2592945-21apt-get08:15:123
252547899161,2cyclictest2550704-21ntpq07:30:233
252547899161,2cyclictest2550704-21ntpq07:30:223
2525478991612,2cyclictest0-21swapper/312:05:003
2525478991610,4cyclictest2811059-21latency_hist11:45:003
252547899160,3cyclictest2858722-21ps12:30:173
252547899160,3cyclictest2608115-21idleruntime-cro08:30:013
252547899160,2cyclictest2768462-21ntpq11:00:233
252547699169,5cyclictest2764590-21tr11:00:152
252547699164,3cyclictest2744827-21irqstats10:40:182
252547699163,3cyclictest2109693-21snmpd08:31:482
2525476991613,1cyclictest0-21swapper/210:35:002
2525476991610,4cyclictest2690840-21latency_hist09:50:012
2525476991610,4cyclictest2608108-21cron08:30:012
2525468991614,2cyclictest60350irq/132-enp0s29f1:rx-009:47:190
2525468991613,2cyclictest2572205-21kworker/u8:3+events_unbound08:33:500
2525468991612,3cyclictest2466571-21kworker/u8:1+flush-259:008:20:000
2525468991610,4cyclictest2670310-21latency_hist09:30:010
252547899151,4cyclictest2867260-21unixbench_multi12:35:243
252547899151,3cyclictest2690868-21systemd-run09:50:013
252547899151,3cyclictest2561487-21grep07:40:253
252547899151,2cyclictest2800988-21apt-get11:35:143
252547899151,2cyclictest2737541-21ntpq10:30:193
252547899151,12cyclictest2816205-21apt11:50:003
252547699159,4cyclictest2805430-21grep11:35:262
252547699159,4cyclictest2779659-21cron11:15:002
252547699159,4cyclictest2717005-21sh10:15:002
252547699159,4cyclictest2650022-21tr09:10:162
252547699159,4cyclictest2639468-21diskstats09:00:152
252547699159,4cyclictest2628705-21sh08:50:002
252547699159,4cyclictest2535234-21grep07:15:242
252547699158,5cyclictest2790987-21grep11:25:142
252547699158,5cyclictest2707364-21irqcore10:05:162
252547699158,5cyclictest2707364-21irqcore10:05:152
252547699158,5cyclictest2675009-21tail09:30:262
252547699158,5cyclictest2622587-21strings08:40:202
252547699158,5cyclictest2552285-21ps07:35:182
252547699157,6cyclictest2816421-21sed11:50:002
252547699157,6cyclictest2597861-21awk08:20:002
252547699157,6cyclictest2577865-21fschecks_time08:00:142
252547699155,2cyclictest2696507-21fw_conntrack09:55:152
252547699154,6cyclictest2617603-21ntp_states08:35:242
252547699154,4cyclictest0-21swapper/209:25:002
252547699154,2cyclictest2785706-21diskstats11:20:142
252547699154,2cyclictest2757969-21memory10:50:202
2525476991511,3cyclictest2705227-21sort10:00:192
2525476991510,4cyclictest2816446-21/usr/sbin/munin11:50:132
2525476991510,3cyclictest335557-21systemd-journal12:10:002
2525476991510,3cyclictest2831627-21cron12:05:002
2525476991510,3cyclictest2561895-21sh07:45:012
2525476991510,3cyclictest0-21swapper/207:30:202
2525476991510,3cyclictest0-21swapper/207:30:192
2525468991513,2cyclictest60350irq/132-enp0s29f1:rx-010:29:180
2525468991513,1cyclictest60350irq/132-enp0s29f1:rx-011:50:140
2525468991513,1cyclictest60350irq/132-enp0s29f1:rx-011:25:150
2525468991511,3cyclictest2628950-21kworker/u8:0+flush-259:009:35:170
252547899140,2cyclictest2525484-21grep07:10:003
252547699148,5cyclictest2634292-21tr08:55:122
252547699148,4cyclictest2842328-21grep12:15:122
252547699146,6cyclictest2821812-21fschecks_time11:55:152
252547699144,2cyclictest2749163-21df_inode10:45:142
2525468991412,1cyclictest60350irq/132-enp0s29f1:rx-012:35:120
2525468991410,3cyclictest2628950-21kworker/u8:0+events_unbound09:30:140
252547699134,2cyclictest2796001-21df_inode11:30:132
2525468991310,2cyclictest60350irq/132-enp0s29f1:rx-007:15:080
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional