You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-25 - 06:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Thu Jun 25, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
487559995651,3cyclictest509-21rs:main1
487571995045,3cyclictest653384-21latency_hist21:50:013
487563995044,4cyclictest767870-21latency_hist23:40:012
487571994741,4cyclictest335557-21systemd-journal20:25:133
487559994539,4cyclictest768104-21/usr/sbin/munin23:40:141
487555994438,4cyclictest335557-21systemd-journal23:05:010
487555994432,10cyclictest335557-21systemd-journal22:00:420
487555994336,5cyclictest548903-21ntpq20:05:210
487563994236,4cyclictest509-21rs:main2
487563994235,5cyclictest335557-21systemd-journal22:33:412
487555994137,2cyclictest211rcuc/021:55:010
487555994129,11cyclictest752055-21gpgv23:25:000
487571994035,4cyclictest658499-21apt-get21:55:003
487559994036,3cyclictest648448-21kworker/u8:3+flush-259:023:25:001
487571993935,3cyclictest648448-21kworker/u8:3+flush-259:022:05:133
487563993935,3cyclictest591209-21kworker/u8:4+flush-259:021:00:192
487563993934,4cyclictest565196-21kworker/u8:1+events_unbound20:30:192
487559993935,3cyclictest555140-21/usr/sbin/munin20:15:151
487571993832,4cyclictest549753-21latency_hist20:10:003
487571993832,4cyclictest544614-21latency_hist20:05:003
487571993732,4cyclictest513772-21cat19:35:013
487559993734,2cyclictest648448-21kworker/u8:3+flush-259:022:15:011
487555993632,3cyclictest690542-21apt-get22:25:120
487555993629,5cyclictest704716-21ntpq22:35:190
487571993532,2cyclictest591209-21kworker/u8:4+flush-259:021:05:003
487571993529,4cyclictest809022-21latency_hist00:20:013
487563993530,3cyclictest621994-21kworker/u8:0+flush-259:021:35:172
487559993530,3cyclictest335557-21systemd-journal00:15:011
487559993529,4cyclictest804145-21/usr/sbin/munin00:15:141
487559993529,4cyclictest335557-21systemd-journal23:40:001
487555993530,3cyclictest560275-21apt-get20:20:120
487563993430,3cyclictest778392-21kworker/u8:2+flush-259:000:00:212
487563993429,4cyclictest621994-21kworker/u8:0+events_unbound21:42:432
487571993329,2cyclictest711012-21kworker/u8:1+flush-259:022:55:003
487563993329,3cyclictest793633-21kworker/u8:1+events_unbound00:27:542
487571993228,3cyclictest680160-21kworker/u8:4+flush-259:022:35:003
487571993228,3cyclictest565196-21kworker/u8:1+events_unbound20:50:183
487571993228,3cyclictest565196-21kworker/u8:1+events_unbound20:50:183
487563993228,3cyclictest648448-21kworker/u8:3+events_unbound22:15:232
487563993228,3cyclictest591209-21kworker/u8:4+flush-259:021:05:202
487563993227,4cyclictest705667-21kworker/u8:0+events_unbound23:46:372
487563993227,3cyclictest580768-21kworker/u8:2+events_unbound21:18:442
487563993226,4cyclictest680159-21kworker/u8:2+events_unbound22:37:442
487559993229,2cyclictest721105-21kworker/u8:2+flush-259:023:10:001
487571993125,4cyclictest513445-21cp19:35:013
487563993128,2cyclictest565196-21kworker/u8:1+flush-259:020:55:432
487563993127,3cyclictest565196-21kworker/u8:1+events_unbound21:55:222
487563993125,4cyclictest757829-21apt-get23:30:102
487563993125,4cyclictest493054-21diskstats19:15:132
487571993026,3cyclictest621994-21kworker/u8:0+events_unbound21:40:163
487563993026,3cyclictest524014-21kworker/u8:3+events_unbound19:55:432
487563992925,3cyclictest621994-21kworker/u8:0+events_unbound21:54:432
487571992824,3cyclictest674367-21kworker/u8:0+flush-259:022:15:423
487563992824,3cyclictest731635-21kworker/u8:1+events_unbound23:20:252
487563992824,3cyclictest565196-21kworker/u8:1+events_unbound21:22:442
487563992824,3cyclictest508326-21kworker/u8:1+events_unbound19:54:422
487563992824,3cyclictest472390-21kworker/u8:4+events_unbound19:13:432
487563992823,4cyclictest793633-21kworker/u8:1+flush-259:000:30:232
487563992823,4cyclictest565196-21kworker/u8:1+events_unbound20:35:132
487555992822,4cyclictest533445-21ntpq19:50:210
487571992723,3cyclictest731635-21kworker/u8:1+events_unbound23:45:153
487563992723,3cyclictest721106-21kworker/u8:4+flush-259:022:55:202
487563992723,3cyclictest680160-21kworker/u8:4+events_unbound22:25:222
487563992723,3cyclictest534362-21kworker/u8:0+flush-259:020:20:152
487563992722,4cyclictest524014-21kworker/u8:3+flush-259:020:10:242
487563992721,4cyclictest674351-21latency_hist22:10:012
487555992721,4cyclictest633020-21apt-get21:30:110
487571992623,2cyclictest524014-21kworker/u8:3+events_unbound20:20:243
487571992623,2cyclictest458196-21kworker/u8:2+events_unbound19:25:203
487563992623,2cyclictest508326-21kworker/u8:1+flush-259:019:40:242
487559992622,3cyclictest778392-21kworker/u8:2+events_unbound00:05:191
487559992622,3cyclictest680159-21kworker/u8:2+events_unbound22:35:151
487571992522,2cyclictest648448-21kworker/u8:3+flush-259:022:20:163
487563992521,3cyclictest705667-21kworker/u8:0+events_unbound23:50:222
487563992521,3cyclictest648448-21kworker/u8:3+events_unbound23:10:152
487559992521,3cyclictest680160-21kworker/u8:4+flush-259:022:30:001
487559992521,3cyclictest565196-21kworker/u8:1+events_unbound21:10:161
487559992521,3cyclictest508326-21kworker/u8:1+flush-259:019:55:231
487559992519,4cyclictest647377-21ntpq21:40:221
487555992518,5cyclictest491749-21ntpq19:10:200
487571992420,3cyclictest555122-21kworker/u8:2+flush-259:020:15:153
487563992421,2cyclictest565196-21kworker/u8:1+events_unbound21:10:222
487563992421,2cyclictest458196-21kworker/u8:2+events_unbound19:45:142
487563992420,3cyclictest705667-21kworker/u8:0+events_unbound23:25:222
487563992420,3cyclictest580768-21kworker/u8:2+events_unbound20:45:222
487559992421,2cyclictest508326-21kworker/u8:1+events_unbound19:40:141
487559992420,3cyclictest705667-21kworker/u8:0+events_unbound22:55:201
487555992420,3cyclictest458196-21kworker/u8:2+events_unbound19:35:010
487555992417,5cyclictest509-21rs:main0
487571992320,2cyclictest591209-21kworker/u8:4+flush-259:021:20:223
487571992319,3cyclictest492637-21latency_hist19:15:013
487571992318,4cyclictest565196-21kworker/u8:1+flush-259:022:10:163
487563992319,3cyclictest731635-21kworker/u8:1+events_unbound23:09:442
487563992319,3cyclictest534362-21kworker/u8:0+flush-259:020:40:202
487563992319,3cyclictest458196-21kworker/u8:2+events_unbound19:30:222
487563992318,3cyclictest731635-21kworker/u8:1+flush-259:023:40:242
487559992319,3cyclictest648448-21kworker/u8:3+flush-259:022:00:251
487571992220,1cyclictest778392-21kworker/u8:2+events_unbound00:00:213
487571992219,3cyclictest534362-21kworker/u8:0+flush-259:020:40:173
487571992219,2cyclictest731635-21kworker/u8:1+flush-259:023:40:013
487571992219,2cyclictest721105-21kworker/u8:2+events_unbound23:00:163
487571992218,3cyclictest731635-21kworker/u8:1+events_unbound23:40:233
487571992217,4cyclictest793847-21/usr/sbin/munin00:05:143
487563992218,3cyclictest524014-21kworker/u8:3+events_unbound20:15:242
487563992218,2cyclictest648448-21kworker/u8:3+events_unbound23:15:472
487563992217,3cyclictest534362-21kworker/u8:0+flush-259:020:25:212
487559992219,2cyclictest793633-21kworker/u8:1+flush-259:000:25:161
487559992219,2cyclictest731635-21kworker/u8:1+flush-259:023:45:141
487559992219,2cyclictest621994-21kworker/u8:0+flush-259:021:35:201
487559992219,2cyclictest590973-21turbostat.cron20:50:001
487559992218,3cyclictest591209-21kworker/u8:4+events_unbound20:55:191
487555992217,3cyclictest565196-21kworker/u8:1+flush-259:021:10:170
487555992217,3cyclictest1-21systemd23:12:420
487555992215,5cyclictest502091-21ntpq19:20:210
487571992118,2cyclictest508326-21kworker/u8:1+events_unbound19:45:233
487571992117,3cyclictest793633-21kworker/u8:1+flush-259:000:20:223
487571992116,3cyclictest736586-21latency_hist23:10:013
487563992117,3cyclictest793633-21kworker/u8:1+events_unbound00:38:002
487563992117,3cyclictest793633-21kworker/u8:1+events_unbound00:13:212
487563992117,3cyclictest721105-21kworker/u8:2+events_unbound23:02:432
487563992115,4cyclictest335557-21systemd-journal00:20:012
487563992113,5cyclictest335557-21systemd-journal22:44:422
487559992118,2cyclictest472390-21kworker/u8:4+flush-259:019:25:161
487559992117,3cyclictest685358-21apt-get22:20:131
487559992117,3cyclictest508326-21kworker/u8:1+events_unbound19:45:221
487555992115,4cyclictest335557-21systemd-journal00:28:340
487571992016,3cyclictest648448-21kworker/u8:3+flush-259:021:55:163
487571992016,3cyclictest648448-21kworker/u8:3+events_unbound23:10:173
487571992016,3cyclictest591209-21kworker/u8:4+flush-259:020:55:153
487571992016,3cyclictest565196-21kworker/u8:1+flush-259:021:35:253
487563992017,2cyclictest648448-21kworker/u8:3+events_unbound22:20:172
487563992017,2cyclictest472390-21kworker/u8:4+events_unbound19:28:442
487563992016,3cyclictest648448-21kworker/u8:3+events_unbound22:00:202
487563992015,4cyclictest507-21in:imuxsock22:49:592
487555992016,3cyclictest487564-21latency_hist19:10:010
487555992014,4cyclictest742011-21apt-get23:15:120
487571991915,3cyclictest731635-21kworker/u8:1+flush-259:023:15:133
487559991916,2cyclictest591209-21kworker/u8:4+flush-259:021:05:251
487559991916,2cyclictest524014-21kworker/u8:3+events_unbound20:05:231
487559991915,3cyclictest565196-21kworker/u8:1+events_unbound21:45:151
487559991915,3cyclictest565196-21kworker/u8:1+events_unbound21:25:201
487559991915,3cyclictest534316-21latency_hist19:55:011
487559991915,3cyclictest458196-21kworker/u8:2+events_unbound19:20:201
487555991916,2cyclictest0-21swapper/023:45:010
487571991815,2cyclictest705667-21kworker/u8:0+flush-259:023:25:153
487571991815,2cyclictest680160-21kworker/u8:4+events_unbound22:25:213
487571991814,3cyclictest565196-21kworker/u8:1+flush-259:020:35:163
487571991814,3cyclictest335557-21systemd-journal21:20:013
487571991811,5cyclictest335557-21systemd-journal20:15:003
487563991815,2cyclictest679840-21latency_hist22:15:022
487563991815,2cyclictest458196-21kworker/u8:2+flush-259:019:20:172
487563991814,3cyclictest508326-21kworker/u8:1+flush-259:020:07:482
487563991813,4cyclictest793633-21kworker/u8:1+flush-259:000:20:152
487559991817,1cyclictest591209-21kworker/u8:4+flush-259:021:20:211
487559991815,2cyclictest658535-21apt-key21:55:001
487559991815,2cyclictest509-21rs:main1
487559991814,3cyclictest648448-21kworker/u8:3+events_unbound22:30:181
487555991814,3cyclictest731635-21kworker/u8:1+flush-259:023:06:120
487555991814,3cyclictest648448-21kworker/u8:3+flush-259:022:50:010
487555991812,4cyclictest643506-21diskstats21:40:140
487571991714,2cyclictest793633-21kworker/u8:1+flush-259:000:25:243
487563991713,3cyclictest778392-21kworker/u8:2+flush-259:023:55:152
487563991713,3cyclictest591209-21kworker/u8:4+events_unbound21:25:432
487563991713,3cyclictest565196-21kworker/u8:1+events_unbound20:52:442
487563991713,3cyclictest565196-21kworker/u8:1+events_unbound20:52:442
487563991713,2cyclictest508326-21kworker/u8:1+events_unbound20:00:212
487559991715,2cyclictest711012-21kworker/u8:1+events_unbound22:50:231
487559991714,2cyclictest632760-21kworker/u8:2+events_unbound21:30:171
487559991713,3cyclictest793633-21kworker/u8:1+events_unbound00:30:161
487559991713,3cyclictest508326-21kworker/u8:1+events_unbound19:40:011
487559991713,2cyclictest591209-21kworker/u8:4+flush-259:021:00:161
487555991714,2cyclictest606533-21cron21:05:000
487555991714,2cyclictest60350irq/132-enp0s29f1:rx-023:45:440
487555991713,3cyclictest705667-21kworker/u8:0+flush-259:023:35:120
487555991713,2cyclictest0-21swapper/020:30:000
487555991713,1cyclictest60350irq/132-enp0s29f1:rx-020:55:130
487571991613,2cyclictest680159-21kworker/u8:2+events_unbound22:35:163
487571991612,3cyclictest524014-21kworker/u8:3+events_unbound19:55:203
487563991613,2cyclictest458196-21kworker/u8:2+events_unbound19:35:432
487559991613,2cyclictest793633-21kworker/u8:1+flush-259:000:35:211
487559991613,2cyclictest580768-21kworker/u8:2+events_unbound20:40:221
487559991613,2cyclictest570326-21idleruntime-cro20:30:001
487559991612,3cyclictest721106-21kworker/u8:4+events_unbound23:00:181
487559991612,3cyclictest674367-21kworker/u8:0+events_unbound22:15:191
487555991614,1cyclictest803905-21idleruntime-cro00:15:010
487555991613,2cyclictest778836-21irqcore23:50:160
487555991613,2cyclictest60350irq/132-enp0s29f1:rx-021:35:200
487555991613,2cyclictest580730-21turbostat20:40:000
487571991512,2cyclictest752228-21gpgv23:25:013
487559991512,1cyclictest0-21swapper/122:45:191
487555991514,1cyclictest60350irq/132-enp0s29f1:rx-021:55:120
487555991513,1cyclictest60350irq/132-enp0s29f1:rx-022:15:200
487555991513,1cyclictest60350irq/132-enp0s29f1:rx-020:30:140
487555991513,1cyclictest60350irq/132-enp0s29f1:rx-020:15:150
487555991513,1cyclictest60350irq/132-enp0s29f1:rx-000:05:180
487555991513,1cyclictest60350irq/132-enp0s29f1:rx-000:00:160
487555991513,1cyclictest0-21swapper/020:55:000
487555991513,1cyclictest0-21swapper/020:55:000
487555991512,2cyclictest61150irq/133-enp0s29f1:tx-023:55:230
487555991512,2cyclictest61150irq/133-enp0s29f1:tx-022:05:170
487555991512,2cyclictest60350irq/132-enp0s29f1:rx-022:30:040
487555991512,2cyclictest60350irq/132-enp0s29f1:rx-019:45:220
487571991411,2cyclictest793633-21kworker/u8:1+flush-259:000:35:183
487571991411,2cyclictest612970-21kthreadcore21:10:173
487571991411,2cyclictest580768-21kworker/u8:2+flush-259:020:45:203
487571991411,2cyclictest472390-21kworker/u8:4+flush-259:019:20:243
487571991410,3cyclictest705667-21kworker/u8:0+events_unbound22:40:143
487571991410,3cyclictest335557-21systemd-journal23:35:003
487563991410,3cyclictest711012-21kworker/u8:1+events_unbound22:50:202
487555991413,1cyclictest60350irq/132-enp0s29f1:rx-021:20:060
487555991413,1cyclictest60350irq/132-enp0s29f1:rx-020:47:400
487555991413,1cyclictest60350irq/132-enp0s29f1:rx-019:55:140
487555991412,2cyclictest61150irq/133-enp0s29f1:tx-021:05:160
487555991412,1cyclictest60350irq/132-enp0s29f1:rx-023:25:230
487555991412,1cyclictest60350irq/132-enp0s29f1:rx-022:14:050
487555991412,1cyclictest60350irq/132-enp0s29f1:rx-021:15:250
487555991411,2cyclictest61150irq/133-enp0s29f1:tx-020:10:260
487555991411,2cyclictest60350irq/132-enp0s29f1:rx-021:25:140
487555991411,2cyclictest60350irq/132-enp0s29f1:rx-020:42:250
487555991411,2cyclictest60350irq/132-enp0s29f1:rx-019:35:230
487555991411,2cyclictest60350irq/132-enp0s29f1:rx-019:25:150
487555991410,2cyclictest726241-21latency_hist23:00:000
48757199139,3cyclictest458196-21kworker/u8:2+flush-259:019:15:163
48756399139,3cyclictest778392-21kworker/u8:2+flush-259:000:05:202
487559991311,1cyclictest0-21swapper/122:00:001
487559991310,2cyclictest705667-21kworker/u8:0+flush-259:022:40:201
48755599138,3cyclictest60350irq/132-enp0s29f1:rx-021:45:120
487555991311,1cyclictest60350irq/132-enp0s29f1:rx-020:00:230
487555991310,2cyclictest60350irq/132-enp0s29f1:rx-022:42:430
48757199129,2cyclictest591209-21kworker/u8:4+events_unbound21:05:243
48757199128,3cyclictest793633-21kworker/u8:1+events_unbound00:30:223
48757199128,3cyclictest648448-21kworker/u8:3+flush-259:022:00:163
487559991210,1cyclictest0-21swapper/119:35:011
487559991210,1cyclictest0-21swapper/119:15:001
48755599129,2cyclictest60350irq/132-enp0s29f1:rx-000:20:110
487555991210,1cyclictest60350irq/132-enp0s29f1:rx-022:20:250
48757199119,1cyclictest0-21swapper/321:30:003
48757199118,2cyclictest793633-21kworker/u8:1+events_unbound00:10:153
48755999118,2cyclictest788850-21cpuspeed_turbos00:00:141
48755999118,2cyclictest778491-21cpuspeed_turbos23:50:121
48755599118,2cyclictest60350irq/132-enp0s29f1:rx-019:40:150
48756399106,3cyclictest621994-21kworker/u8:0+flush-259:021:45:262
48755999107,2cyclictest0-21swapper/100:25:001
4875719994,3cyclictest530032-21sed19:50:163
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional