You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-02 - 20:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Thu Apr 02, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3469893995349,3cyclictest3470737-21kworker/u8:4+events_unbound07:15:111
3469893995348,2cyclictest3686704-21kworker/u8:1+events_unbound11:15:231
3469893995046,3cyclictest3712719-21kworker/u8:2+events_unbound12:00:241
3469902994946,2cyclictest3650818-21kworker/u8:3+events_unbound11:25:253
3469893994946,2cyclictest3475971-21kworker/u8:1+flush-259:008:35:241
3469885994942,3cyclictest3541996-21kworker/u8:0+events_unbound09:00:140
3469902994744,3cyclictest3500272-21kworker/u8:4+events_unbound08:50:163
3469893994642,2cyclictest3579668-21kworker/u8:0+events_unbound10:40:121
3469885994642,3cyclictest3650818-21kworker/u8:3+events_unbound11:50:190
3469885994639,2cyclictest3500272-21kworker/u8:4+events_unbound10:00:140
3469893994541,3cyclictest3739195-21apt-get11:55:111
3469885994541,3cyclictest3579668-21kworker/u8:0+events_unbound10:25:230
3469902994441,2cyclictest3475971-21kworker/u8:1+events_unbound08:45:123
3469902994441,2cyclictest3470737-21kworker/u8:4+events_unbound07:15:183
3469893994438,3cyclictest3650818-21kworker/u8:3+events_unbound11:45:231
3469902994340,2cyclictest3650818-21kworker/u8:3+events_unbound12:25:223
3469902994339,3cyclictest3765111-21kworker/u8:0+events_unbound12:35:233
3469902994339,3cyclictest3666326-21kworker/u8:2+events_unbound11:00:233
3469902994339,3cyclictest3666288-21latency_hist10:45:013
3469885994340,2cyclictest3500272-21kworker/u8:4+events_unbound10:15:210
3469902994239,2cyclictest3475971-21kworker/u8:1+events_unbound08:40:233
3469893994239,2cyclictest3770031-21kworker/u8:1+events_unbound12:25:201
3469893994238,3cyclictest3579668-21kworker/u8:0+flush-259:009:45:211
3469885994238,3cyclictest3619880-21kworker/u8:1+flush-259:010:05:240
3469902994138,2cyclictest3475971-21kworker/u8:1+events_unbound07:25:103
3469902994137,3cyclictest3500272-21kworker/u8:4+events_unbound09:30:233
3469893994139,2cyclictest3686704-21kworker/u8:1+events_unbound11:25:151
3469902994037,2cyclictest3712719-21kworker/u8:2+events_unbound11:35:243
3469893994037,2cyclictest3579668-21kworker/u8:0+events_unbound10:50:151
3469893994035,3cyclictest3614702-21latency_hist09:55:001
3469885994037,2cyclictest3650818-21kworker/u8:3+events_unbound10:30:180
3469885994036,3cyclictest3749194-21kworker/u8:4+events_unbound12:15:210
3469885994036,3cyclictest3500272-21kworker/u8:4+events_unbound08:55:130
3469893993936,2cyclictest3475971-21kworker/u8:1+events_unbound07:25:091
3469893993935,3cyclictest3500272-21kworker/u8:4+events_unbound08:10:151
3469893993935,2cyclictest3579668-21kworker/u8:0+events_unbound10:00:191
3469885993936,2cyclictest3712719-21kworker/u8:2+events_unbound11:35:220
3469902993836,2cyclictest3500272-21kworker/u8:4+events_unbound09:50:223
3469902993835,2cyclictest3697296-21kworker/u8:0+events_unbound11:20:213
3469902993835,2cyclictest3650818-21kworker/u8:3+events_unbound10:50:223
3469893993835,2cyclictest3650818-21kworker/u8:3+events_unbound12:15:131
3469893993834,3cyclictest3500272-21kworker/u8:4+events_unbound08:05:171
3469902993734,2cyclictest3770031-21kworker/u8:1+events_unbound12:30:143
3469902993734,2cyclictest3650818-21kworker/u8:3+events_unbound10:30:253
3469893993734,2cyclictest3697296-21kworker/u8:0+events_unbound11:20:131
3469893993734,2cyclictest3562724-21kworker/u8:1+events_unbound09:15:161
3469885993734,2cyclictest3712719-21kworker/u8:2+events_unbound12:00:220
3469885993734,2cyclictest3712719-21kworker/u8:2+events_unbound11:30:140
3469885993734,2cyclictest3692047-21kworker/u8:2+events_unbound11:10:190
3469885993734,2cyclictest3475971-21kworker/u8:1+events_unbound07:55:200
3469885993733,3cyclictest3770031-21kworker/u8:1+events_unbound12:30:120
3469885993733,3cyclictest3463543-21kworker/u8:3+events_unbound07:12:230
3469893993633,2cyclictest3500272-21kworker/u8:4+flush-259:008:50:231
3469893993632,3cyclictest3475971-21kworker/u8:1+events_unbound08:20:191
3469885993629,5cyclictest3650818-21kworker/u8:3+flush-259:011:00:170
3469902993531,3cyclictest400374-21kworker/u8:0+events_unbound07:40:013
3469893993532,2cyclictest3765111-21kworker/u8:0+events_unbound12:35:121
3469893993532,2cyclictest3686704-21kworker/u8:1+events_unbound11:40:141
3469885993532,2cyclictest3650818-21kworker/u8:3+flush-259:010:55:210
3469885993532,2cyclictest3475971-21kworker/u8:1+events_unbound07:35:170
3469885993530,2cyclictest3765111-21kworker/u8:0+events_unbound12:35:140
3469893993431,2cyclictest3541996-21kworker/u8:0+events_unbound09:05:141
3469893993430,3cyclictest3692047-21kworker/u8:2+flush-259:011:10:191
3469893993430,3cyclictest3619880-21kworker/u8:1+events_unbound10:20:131
3469885993431,2cyclictest3650818-21kworker/u8:3+flush-259:011:25:200
3469885993431,2cyclictest3579668-21kworker/u8:0+flush-259:011:05:250
3469885993430,3cyclictest3500272-21kworker/u8:4+flush-259:009:35:230
3469885993430,3cyclictest3500272-21kworker/u8:4+events_unbound08:30:240
3469902993330,2cyclictest3463543-21kworker/u8:3+events_unbound07:13:573
3469902993328,2cyclictest3712719-21kworker/u8:2+events_unbound11:45:123
3469893993330,2cyclictest3541996-21kworker/u8:0+flush-259:009:00:161
3469893993329,3cyclictest3579668-21kworker/u8:0+events_unbound10:25:151
3469893993329,3cyclictest3475971-21kworker/u8:1+events_unbound07:55:181
3469885993330,2cyclictest3541996-21kworker/u8:0+events_unbound09:05:200
3469885993329,3cyclictest3697296-21kworker/u8:0+events_unbound11:20:170
3469885993328,2cyclictest3579668-21kworker/u8:0+flush-259:010:45:170
3469885993326,5cyclictest3727733-21ntpq11:40:200
3469893993229,2cyclictest3551895-21kworker/u8:3+events_unbound09:10:221
3469893993229,2cyclictest3488184-21kworker/u8:4+events_unbound07:45:181
3469893993229,2cyclictest3475971-21kworker/u8:1+events_unbound08:45:151
3469885993229,2cyclictest3579668-21kworker/u8:0+flush-259:009:30:180
3469885993228,3cyclictest400374-21kworker/u8:0+events_unbound08:05:090
3469902993128,3cyclictest3712719-21kworker/u8:2+events_unbound12:04:193
3469902993128,2cyclictest3573473-21kworker/u8:2+events_unbound09:15:253
3469897993127,3cyclictest3703096-21/usr/sbin/munin11:20:142
3469897993127,3cyclictest252-21systemd-journal07:58:562
3469893993128,2cyclictest3500272-21kworker/u8:4+events_unbound10:10:221
3469885993128,2cyclictest3579668-21kworker/u8:0+events_unbound09:20:210
3469885993127,3cyclictest3470274-21kworker/u8:2+flush-259:008:10:280
3469902993027,2cyclictest400374-21kworker/u8:0+events_unbound07:20:523
3469902993027,2cyclictest3541996-21kworker/u8:0+events_unbound09:08:023
3469897993026,3cyclictest3755046-21/usr/sbin/munin12:10:122
3469893993026,3cyclictest3650818-21kworker/u8:3+events_unbound12:10:171
3469885993027,2cyclictest3650818-21kworker/u8:3+events_unbound10:50:210
3469885993026,3cyclictest3500272-21kworker/u8:4+events_unbound08:50:210
3469885993025,2cyclictest3500272-21kworker/u8:4+events_unbound10:10:240
3469902992927,2cyclictest3541996-21kworker/u8:0+events_unbound08:55:263
3469902992926,2cyclictest3593944-21kworker/u8:2+events_unbound09:38:003
3469902992926,2cyclictest3500272-21kworker/u8:4+events_unbound09:55:123
3469902992925,3cyclictest3650818-21kworker/u8:3+events_unbound12:12:203
3469902992924,3cyclictest252-21systemd-journal08:24:593
3469893992926,2cyclictest3579668-21kworker/u8:0+events_unbound10:15:261
3469893992926,2cyclictest3500272-21kworker/u8:4+events_unbound08:55:111
3469893992926,2cyclictest3475971-21kworker/u8:1+events_unbound08:30:141
3469893992925,3cyclictest3475971-21kworker/u8:1+events_unbound08:25:221
3469893992925,3cyclictest3475971-21kworker/u8:1+events_unbound08:25:211
3469885992926,2cyclictest3579668-21kworker/u8:0+flush-259:009:25:200
3469902992826,2cyclictest3579668-21kworker/u8:0+events_unbound11:05:203
3469902992825,2cyclictest3697296-21kworker/u8:0+events_unbound11:15:493
3469902992824,3cyclictest3475971-21kworker/u8:1+events_unbound07:49:573
3469893992825,2cyclictest3738714-21kworker/u8:0+flush-259:012:05:211
3469893992825,2cyclictest3650818-21kworker/u8:3+events_unbound10:45:111
3469893992825,2cyclictest3579668-21kworker/u8:0+events_unbound11:05:241
3469893992824,3cyclictest252-21systemd-journal07:12:221
3469885992825,2cyclictest3551895-21kworker/u8:3+events_unbound09:10:210
3469902992725,2cyclictest3749194-21kworker/u8:4+events_unbound12:15:233
3469902992725,2cyclictest3470274-21kworker/u8:2+events_unbound08:13:583
3469902992724,3cyclictest3500272-21kworker/u8:4+events_unbound08:07:243
3469902992721,5cyclictest3530172-21ntpq08:30:213
3469893992724,2cyclictest3666326-21kworker/u8:2+events_unbound11:00:181
3469902992623,3cyclictest3500272-21kworker/u8:4+events_unbound10:07:023
3469902992623,2cyclictest3686704-21kworker/u8:1+events_unbound11:30:503
3469902992623,2cyclictest3500272-21kworker/u8:4+events_unbound09:27:003
3469902992623,2cyclictest3475971-21kworker/u8:1+events_unbound07:50:283
3469893992623,2cyclictest3650818-21kworker/u8:3+events_unbound10:30:171
3469893992623,2cyclictest3475971-21kworker/u8:1+events_unbound08:15:241
3469893992622,3cyclictest3650818-21kworker/u8:3+events_unbound10:35:261
3469885992623,2cyclictest3749194-21kworker/u8:4+events_unbound12:10:110
3469885992623,2cyclictest3593944-21kworker/u8:2+events_unbound09:40:260
3469885992623,2cyclictest3475971-21kworker/u8:1+flush-259:008:40:250
3469902992523,2cyclictest3579668-21kworker/u8:0+events_unbound10:45:223
3469902992522,2cyclictest400374-21kworker/u8:0+events_unbound07:59:573
3469902992522,2cyclictest3579668-21kworker/u8:0+events_unbound10:39:003
3469902992522,2cyclictest3500272-21kworker/u8:4+events_unbound08:19:593
3469893992522,2cyclictest3579668-21kworker/u8:0+flush-259:009:40:181
3469885992522,2cyclictest3749194-21kworker/u8:4+events_unbound12:05:280
3469885992522,2cyclictest3579668-21kworker/u8:0+flush-259:009:45:240
3469902992421,2cyclictest3712719-21kworker/u8:2+events_unbound11:59:563
3469902992421,2cyclictest3475971-21kworker/u8:1+events_unbound08:25:023
3469902992421,2cyclictest3475971-21kworker/u8:1+events_unbound08:25:013
3469902992421,1cyclictest400374-21kworker/u8:0+events_unbound08:04:563
3469897992418,4cyclictest252-21systemd-journal09:25:202
3469897992417,5cyclictest3738923-21latency_hist11:55:012
3469897992417,5cyclictest252-21systemd-journal11:00:002
3469893992421,2cyclictest3650818-21kworker/u8:3+flush-259:012:21:201
3469893992421,2cyclictest3500272-21kworker/u8:4+flush-259:010:05:001
3469885992421,2cyclictest3749194-21kworker/u8:4+flush-259:012:20:220
3469885992420,2cyclictest400374-21kworker/u8:0+events_unbound07:34:570
3469902992321,2cyclictest3562724-21kworker/u8:1+events_unbound09:13:003
3469902992320,2cyclictest3686704-21kworker/u8:1+events_unbound11:10:023
3469902992320,2cyclictest3500272-21kworker/u8:4+events_unbound08:36:213
346989799234,7cyclictest3610116-21hddtemp_smartct09:50:142
3469893992321,2cyclictest3770031-21kworker/u8:1+events_unbound12:30:021
3469893992321,2cyclictest3475971-21kworker/u8:1+events_unbound07:40:571
3469893992318,2cyclictest400374-21kworker/u8:0+flush-259:007:39:211
3469885992319,3cyclictest3475971-21kworker/u8:1+events_unbound08:25:250
3469885992319,3cyclictest3475971-21kworker/u8:1+events_unbound08:25:250
3469902992220,2cyclictest3686704-21kworker/u8:1+events_unbound11:40:513
3469902992219,2cyclictest3500272-21kworker/u8:4+events_unbound10:10:113
346989799225,10cyclictest3598204-21ntpq09:35:192
346989799224,5cyclictest3701957-21ntpq11:15:182
3469897992216,4cyclictest3541327-21latency_hist08:45:002
3469893992220,2cyclictest3579668-21kworker/u8:0+events_unbound09:30:211
3469893992219,2cyclictest3686704-21kworker/u8:1+events_unbound11:35:281
3469885992219,2cyclictest400374-21kworker/u8:0+events_unbound07:54:550
3469885992217,3cyclictest252-21systemd-journal07:20:520
3469902992119,2cyclictest3579668-21kworker/u8:0+events_unbound10:25:033
3469902992119,2cyclictest3579668-21kworker/u8:0+events_unbound09:21:413
3469902992118,2cyclictest3640042-21kworker/u8:2+events_unbound10:16:193
3469902992118,2cyclictest3541996-21kworker/u8:0+events_unbound09:01:003
346989799216,4cyclictest3780161-21ntpq12:30:212
3469897992115,5cyclictest3774512-21ntpq12:25:182
3469897992115,4cyclictest3562181-21gpgv09:05:012
3469897992114,5cyclictest3651060-21apt-get10:30:102
3469893992119,2cyclictest3500272-21kworker/u8:4+events_unbound09:20:111
3469893992116,2cyclictest3470274-21kworker/u8:2+events_unbound07:32:561
3469885992119,2cyclictest3488184-21kworker/u8:4+events_unbound07:45:200
3469885992118,2cyclictest3712719-21kworker/u8:2+events_unbound11:55:170
3469885992118,2cyclictest3470274-21kworker/u8:2+events_unbound07:17:570
3469885992114,5cyclictest3644790-21ntpq10:20:200
3469902992017,2cyclictest3749915-21apt-get12:05:133
346989799205,8cyclictest3635374-21munin-run10:15:002
346989799204,5cyclictest3525245-21sed08:25:242
346989799204,5cyclictest3525245-21sed08:25:232
346989799204,4cyclictest3514914-21grep08:15:232
3469897992014,4cyclictest3493781-21latency_hist07:55:002
3469897992014,3cyclictest252-21systemd-journal07:41:552
3469893992017,2cyclictest3712719-21kworker/u8:2+events_unbound11:53:311
3469893992017,2cyclictest3500272-21kworker/u8:4+flush-259:009:55:061
3469893992015,3cyclictest509-21rs:main1
3469885992018,1cyclictest3470274-21kworker/u8:2+events_unbound08:20:090
3469885992017,2cyclictest3650818-21kworker/u8:3+events_unbound11:45:550
3469885992016,3cyclictest400374-21kworker/u8:0+events_unbound07:29:580
3469885992016,3cyclictest3770031-21kworker/u8:1+flush-259:012:26:230
3469902991918,1cyclictest3579668-21kworker/u8:0+events_unbound09:47:213
3469902991917,2cyclictest3593944-21kworker/u8:2+events_unbound09:42:013
3469902991916,2cyclictest400374-21kworker/u8:0+events_unbound07:39:003
3469902991916,2cyclictest3579668-21kworker/u8:0+events_unbound10:55:053
346989799195,7cyclictest3696367-21ntpq11:10:202
346989799195,4cyclictest3615124-21diskstats09:55:112
346989799195,4cyclictest3557090-21date09:00:002
346989799193,5cyclictest3636102-21cut10:15:172
3469893991916,2cyclictest400374-21kworker/u8:0+events_unbound08:00:071
3469885991917,2cyclictest3470274-21kworker/u8:2+events_unbound08:00:550
3469885991916,2cyclictest3579668-21kworker/u8:0+events_unbound09:50:150
3469885991916,2cyclictest3500272-21kworker/u8:4+events_unbound08:37:490
3469885991916,2cyclictest3500272-21kworker/u8:4+events_unbound08:17:430
3469902991816,2cyclictest400374-21kworker/u8:0+events_unbound07:31:593
3469902991816,2cyclictest3650818-21kworker/u8:3+events_unbound12:21:223
3469902991815,2cyclictest3712719-21kworker/u8:2+events_unbound11:52:243
346989799188,2cyclictest3670487-21ntp_10:45:192
346989799185,3cyclictest3509540-21ntpq08:10:232
346989799184,4cyclictest3671838-21df_inode10:50:132
346989799183,5cyclictest3504309-21ntpq08:05:162
346989799183,4cyclictest3482732-21smartctl07:35:102
3469897991814,2cyclictest3579668-21kworker/u8:0+events_unbound11:05:182
3469897991812,4cyclictest3531016-21latency_hist08:35:002
3469885991814,3cyclictest3579668-21kworker/u8:0+events_unbound10:39:030
346990299176,1cyclictest191rcu_preempt10:00:143
346989799177,5cyclictest3579665-21irqstats09:20:162
346989799176,6cyclictest3470531-21systemd-run07:15:002
346989799175,6cyclictest3781412-21df12:35:132
346989799175,4cyclictest3723070-21vmstat11:35:252
346989799174,7cyclictest3470923-21diskstats07:15:102
346989799174,2cyclictest3603427-21ntpq09:40:192
346989799172,5cyclictest3620092-21/usr/sbin/munin10:00:222
346989799172,4cyclictest3608639-21ldconfig09:45:192
3469893991714,2cyclictest3475971-21kworker/u8:1+events_unbound07:53:561
3469885991713,3cyclictest400374-21kworker/u8:0+events_unbound07:41:450
3469885991713,3cyclictest3579668-21kworker/u8:0+events_unbound10:40:140
346989799165,6cyclictest3708438-21df_inode11:25:152
346989799165,2cyclictest3665206-21memory10:40:182
346989799164,7cyclictest3729178-21fw_conntrack11:45:142
346989799164,3cyclictest3748359-21ntpq12:00:212
346989799164,2cyclictest3481524-21munin-run07:29:592
346989799163,4cyclictest3550790-21ntpq08:50:212
3469893991614,2cyclictest3470274-21kworker/u8:2+events_unbound07:23:561
3469893991613,2cyclictest3500272-21kworker/u8:4+flush-259:009:37:251
3469893991613,2cyclictest3475971-21kworker/u8:1+flush-259:008:42:201
3469885991614,1cyclictest60350irq/132-enp0s29f1:rx-008:45:170
3469885991613,2cyclictest3697296-21kworker/u8:0+flush-259:011:16:130
3469885991613,2cyclictest3562724-21kworker/u8:1+events_unbound09:17:410
346989799157,6cyclictest3754797-21cron12:10:012
346989799157,6cyclictest3573233-21idleruntime-cro09:15:002
346989799155,5cyclictest3646080-21df_inode10:25:132
346989799155,2cyclictest3577346-21memory09:15:182
346989799154,3cyclictest3739349-21df_inode11:55:142
346989799154,2cyclictest3660261-21ntpq10:35:202
346989799153,3cyclictest3535311-21gcc08:35:212
3469897991511,2cyclictest3568048-21latency_hist09:10:012
3469897991510,3cyclictest3499615-21apt08:05:002
346989799148,4cyclictest3482177-21munin-plugin-st07:35:002
346989799147,5cyclictest3475968-21time-dir07:20:512
346989799144,6cyclictest832-21runrttasks09:30:112
346989799144,5cyclictest3625603-21fw_forwarded_lo10:05:142
346989799144,5cyclictest3516809-21irqstats08:20:162
346989799144,2cyclictest3765512-21diskstats12:20:132
346989799144,2cyclictest3542574-21irqstats08:45:152
346989799142,4cyclictest3492239-21gcc07:45:172
3469897991411,2cyclictest3760653-21irqcore12:15:132
3469897991411,2cyclictest3640971-21fschecks_time10:20:122
3469893991412,2cyclictest3666326-21kworker/u8:2+flush-259:010:55:371
346989799137,4cyclictest3713524-21cpuspeed_turbos11:30:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional