You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-14 - 22:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Mon Sep 14, 2026 12:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
218012239166,24sleep30-21swapper/307:07:173
217642239163,26sleep10-21swapper/107:06:471
218552227155,59sleep00-21swapper/007:07:590
217132202166,24sleep20-21swapper/207:06:082
2138021830,4sleep22211199cyclictest12:00:132
1422421430,5sleep12210599cyclictest11:44:591
256672980,5sleep02210199cyclictest08:30:010
65512460,3sleep011-21rcuc/007:45:150
25862460,2sleep20-21swapper/210:04:392
2210199368,13cyclictest18022-21basename10:40:010
22101993512,12cyclictest1382-21awk07:35:010
224532310,4sleep20-21swapper/208:20:202
2210199315,13cyclictest15819-21munin-run10:35:010
273132300,4sleep011-21rcuc/008:30:260
2210199309,18cyclictest5455-21awk11:25:010
22111992917,3cyclictest0-21swapper/212:09:552
2211199291,12cyclictest0-21swapper/209:46:192
2211199291,12cyclictest0-21swapper/207:11:392
2210199296,11cyclictest7036-21perf10:15:000
2210199291,13cyclictest11421-21munin-run10:25:000
225192280,3sleep22211199cyclictest10:50:022
22111992817,3cyclictest0-21swapper/210:14:472
2210199283,12cyclictest13008-21munin-run09:15:010
2211199271,13cyclictest0-21swapper/209:56:592
2210599271,17cyclictest5723-21chrt07:44:281
2210199273,12cyclictest23059-21awk12:05:010
2210199271,13cyclictest3877-21awk12:35:010
2210199270,19cyclictest0-21swapper/009:51:390
2211199261,13cyclictest0-21swapper/212:10:112
22111992610,3cyclictest0-21swapper/210:39:432
2211199260,10cyclictest0-21swapper/210:17:092
22105992618,5cyclictest8024-21/usr/sbin/munin10:15:161
2210599260,19cyclictest0-21swapper/111:39:191
2210199262,11cyclictest10808-21diskmemload10:50:010
246232251,3sleep124834-21df_inode07:15:031
2210199257,15cyclictest2004-21munin-run08:50:010
2210199251,12cyclictest24276-21munin-run07:15:010
2211799241,5cyclictest19207-21/usr/sbin/munin08:15:123
2211799241,5cyclictest19207-21/usr/sbin/munin08:15:123
2211799241,21cyclictest0-21swapper/308:20:143
2211799241,12cyclictest21056-21/usr/sbin/munin12:00:243
2211799241,11cyclictest31438-21/usr/sbin/munin11:10:183
2211199241,11cyclictest0-21swapper/208:58:192
2211199240,17cyclictest21236-21wc08:20:012
2211199240,17cyclictest21236-21wc08:20:012
22105992414,3cyclictest5352-21smart_sda12:35:241
2210599241,18cyclictest0-21swapper/112:09:021
2210599241,17cyclictest21675-21kworker/u8:209:40:131
2210599241,16cyclictest0-21swapper/111:27:191
2210599241,14cyclictest0-21swapper/112:32:491
2210199249,9cyclictest81ktimersoftd/012:10:140
2210199247,8cyclictest31438-21/usr/sbin/munin11:10:140
2211799239,4cyclictest27430-21crond12:15:013
2211799232,10cyclictest28612-21/usr/sbin/munin09:50:143
2211799231,20cyclictest0-21swapper/308:55:183
2211799231,11cyclictest29859-21/usr/sbin/munin12:20:223
2211799231,11cyclictest26414-21/usr/sbin/munin09:45:213
2211799231,11cyclictest25801-21/usr/sbin/munin08:30:263
2211199234,4cyclictest0-21swapper/211:01:552
2211199233,6cyclictest0-21swapper/212:35:122
2211199230,12cyclictest0-21swapper/211:32:592
2210599239,3cyclictest20340-21sed11:55:251
22105992317,3cyclictest3511-21switchtime08:50:231
2210599231,4cyclictest16486-21cut11:50:011
2210599231,4cyclictest16486-21cut11:50:011
22105992314,2cyclictest0-21swapper/110:10:011
2210599231,19cyclictest18306-21switchtime08:10:251
2210599231,16cyclictest0-21swapper/107:30:111
2210599231,11cyclictest0-21swapper/108:40:171
2210599230,6cyclictest0-21swapper/110:57:001
2210199237,3cyclictest0-21swapper/012:35:130
2210199231,13cyclictest0-21swapper/009:38:400
111682230,4sleep339-21rcuc/307:55:163
2211799226,4cyclictest31117-21NetworkManager11:08:563
2211799221,19cyclictest0-21swapper/310:24:303
2211799221,13cyclictest87350irq/126-enp1s0-08:05:273
2211799221,11cyclictest839-21lldpd11:48:023
2211799221,11cyclictest839-21lldpd11:48:023
2211799221,10cyclictest23605-21/usr/sbin/munin08:25:133
2211799221,10cyclictest1896-21/usr/sbin/munin12:30:183
2211799221,10cyclictest0-21swapper/310:35:143
2211799221,10cyclictest0-21swapper/309:05:223
2211799220,20cyclictest0-21swapper/311:30:173
2211799220,20cyclictest0-21swapper/309:35:243
2211799220,20cyclictest0-21swapper/307:15:213
2211199223,6cyclictest0-21swapper/211:26:072
2211199221,9cyclictest0-21swapper/208:32:512
2211199221,9cyclictest0-21swapper/207:38:192
2211199221,6cyclictest0-21swapper/208:38:452
22111992211,3cyclictest0-21swapper/209:25:072
2211199221,12cyclictest0-21swapper/208:05:232
2210599225,9cyclictest0-21swapper/109:28:541
22105992218,2cyclictest1996-21sh08:50:011
22105992217,2cyclictest28105-21ntp_states11:00:221
22105992216,3cyclictest23373-21latency_hist08:25:021
2210599221,4cyclictest31084-21expr09:55:141
22105992214,6cyclictest29374-21kworker/u8:112:20:171
2210599221,3cyclictest13234-21users10:25:261
2210599221,19cyclictest0-21swapper/110:35:181
2210599221,18cyclictest14079-21latency09:15:171
2210599221,18cyclictest12987-21fschecks_count08:00:151
2210599221,16cyclictest0-21swapper/109:35:131
2210599221,14cyclictest0-21swapper/111:14:461
2210599221,13cyclictest0-21swapper/111:33:481
2210599221,10cyclictest24479-21swap07:15:021
2210199225,5cyclictest28034-21sort12:15:150
2210199222,5cyclictest89250irq/135-enp2s0-11:08:330
2210199221,4cyclictest10807-21munin-run09:10:010
2210199220,13cyclictest5667-21/usr/sbin/munin11:25:170
178182221,3sleep317920-21memory08:10:193
2211799217,3cyclictest22301-21/usr/sbin/munin07:10:183
2211799215,4cyclictest8564-1kworker/u9:110:53:113
2211799212,7cyclictest26190-21munin-run09:45:003
2211799211,11cyclictest0-21swapper/310:56:363
2211799211,10cyclictest839-21lldpd12:27:023
2211799210,20cyclictest0-21swapper/310:05:183
2211799210,19cyclictest0-21swapper/311:15:203
2211199219,3cyclictest0-21swapper/211:46:112
2211199219,3cyclictest0-21swapper/211:46:112
2211199216,3cyclictest21657-21taskset09:33:312
2211199215,2cyclictest0-21swapper/212:29:352
2211199213,7cyclictest30669-21irqstats12:20:172
2211199211,3cyclictest0-21swapper/210:31:242
22111992112,3cyclictest0-21swapper/211:16:472
2210599216,9cyclictest0-21swapper/111:22:551
2210599216,9cyclictest0-21swapper/109:10:311
2210599215,9cyclictest10808-21diskmemload10:03:131
2210599214,10cyclictest0-21swapper/110:51:171
22105992117,3cyclictest0-21swapper/112:00:211
22105992117,3cyclictest0-21swapper/108:18:051
22105992117,3cyclictest0-21swapper/108:18:051
22105992115,3cyclictest10693-21awk10:20:231
2210599211,4cyclictest0-21swapper/112:13:401
2210599211,3cyclictest0-21swapper/107:50:401
2210599211,18cyclictest0-21swapper/111:07:211
2210599211,15cyclictest0-21swapper/112:16:411
2210599211,15cyclictest0-21swapper/109:53:061
2210599211,15cyclictest0-21swapper/109:32:511
2210599211,15cyclictest0-21swapper/107:38:441
2210599211,15cyclictest0-21swapper/107:21:161
2210599211,13cyclictest0-21swapper/110:45:311
2210599211,12cyclictest26615-21irqstats08:30:171
2210199216,7cyclictest23268-21/usr/sbin/munin12:05:160
2210199216,6cyclictest0-21swapper/009:33:420
2210199215,7cyclictest29859-21/usr/sbin/munin12:20:120
2210199213,8cyclictest1258-21/usr/sbin/munin11:15:270
2210199213,8cyclictest10069-21/usr/sbin/munin11:35:270
2210199212,10cyclictest12607-21/usr/sbin/munin08:00:220
2210199210,13cyclictest0-21swapper/010:54:140
2211799208,2cyclictest0-21swapper/311:20:213
2211799207,3cyclictest839-21lldpd09:22:003
2211799206,3cyclictest808-21snmpd09:57:283
2211799206,3cyclictest6122-21sadc10:10:213
2211799205,3cyclictest87350irq/126-enp1s0-11:50:143
2211799205,3cyclictest87350irq/126-enp1s0-07:30:223
2211799204,3cyclictest10808-21diskmemload12:05:123
2211799202,6cyclictest0-21swapper/309:25:143
2211799201,6cyclictest28298-21smart_sda11:00:243
2211799201,5cyclictest579-21systemd-journal09:35:003
22117992013,4cyclictest11140-21ntp_states11:35:233
2211799201,17cyclictest0-21swapper/308:45:143
2211799200,10cyclictest0-21swapper/308:42:383
2211199207,3cyclictest0-21swapper/210:41:292
2211199207,3cyclictest0-21swapper/207:45:112
2211199205,2cyclictest0-21swapper/207:58:552
2211199204,8cyclictest13849-21which08:00:232
2211199204,2cyclictest87450irq/127-enp1s0-12:32:382
2211199203,3cyclictest0-21swapper/210:25:192
2211199201,9cyclictest15054-21runrttasks09:18:182
2211199201,8cyclictest28008-21df_inode12:15:142
2211199201,12cyclictest0-21swapper/208:26:392
22111992010,2cyclictest0-21swapper/207:15:232
2210599205,4cyclictest28914-21memory08:35:191
2210599204,9cyclictest0-21swapper/110:13:131
22105992016,3cyclictest0-21swapper/111:54:351
22105992016,3cyclictest0-21swapper/109:04:091
22105992015,3cyclictest0-21swapper/109:20:021
22105992015,2cyclictest26762-21diskstats09:45:141
2210599201,12cyclictest0-21swapper/111:16:091
2210599201,12cyclictest0-21swapper/110:40:581
2210599201,12cyclictest0-21swapper/110:32:481
2210599200,5cyclictest0-21swapper/112:28:551
2210199205,6cyclictest14814-21/usr/sbin/munin08:05:120
2210199205,6cyclictest0-21swapper/010:18:030
2210199203,9cyclictest24839-21/usr/sbin/munin10:55:200
2210199202,9cyclictest0-21swapper/009:00:250
2210199201,11cyclictest909-21sssd_nss09:20:250
2210199201,11cyclictest8206-21/usr/sbin/munin07:50:140
2210199201,11cyclictest5584-21sshd07:42:380
2210199201,10cyclictest14464-21/usr/sbin/munin11:45:200
2210199201,10cyclictest14464-21/usr/sbin/munin11:45:200
2210199200,6cyclictest0-21swapper/009:27:560
2210199200,15cyclictest0-21swapper/011:55:120
276342190,4sleep081ktimersoftd/009:45:220
2211799195,3cyclictest87350irq/126-enp1s0-11:25:113
2211799195,3cyclictest87350irq/126-enp1s0-07:35:173
2211799194,3cyclictest29584-21/usr/sbin/munin07:25:163
2211799191,7cyclictest9216-21sh10:20:013
2211799191,6cyclictest27997-21/usr/sbin/munin08:35:263
2211799191,6cyclictest0-21swapper/310:00:453
2211799191,5cyclictest9765-21grep07:50:253
22117991912,4cyclictest4947-21head07:40:233
2211799191,15cyclictest13676-21ntp_states08:00:223
2211799190,9cyclictest0-21swapper/310:48:063
2211799190,1cyclictest0-21swapper/307:45:243
2211199197,2cyclictest10808-21diskmemload09:50:112
2211199196,2cyclictest5040-21gunzip07:40:232
2211199195,3cyclictest25649-21irqstats10:55:162
2211199195,3cyclictest0-21swapper/211:54:192
2211199194,8cyclictest0-21swapper/209:40:592
2211199192,14cyclictest31595-21tr11:10:112
2211199190,9cyclictest0-21swapper/209:00:182
2210599195,8cyclictest0-21swapper/108:07:361
2210599195,3cyclictest23674-21cpuspeed_turbos08:25:131
2210599191,16cyclictest0-21swapper/107:25:381
2210599191,10cyclictest9896-21ntpq09:05:211
2210599190,18cyclictest0-21swapper/107:55:121
2210199198,3cyclictest30812-21/usr/sbin/munin09:55:130
2210199198,3cyclictest12266-21/usr/sbin/munin11:40:240
2210199193,6cyclictest0-21swapper/010:03:420
2210199193,10cyclictest0-21swapper/009:40:390
2210199191,10cyclictest4424-21/usr/sbin/munin08:55:230
2210199191,10cyclictest10409-21/usr/sbin/munin07:55:240
2210199191,10cyclictest0-21swapper/012:26:180
2211799181,6cyclictest20451-21vmstat11:55:263
2211799181,5cyclictest2225-21/usr/sbin/munin08:50:183
2211799181,5cyclictest12317-21hddtemp_smartct10:25:163
2211799181,5cyclictest11031-21/usr/sbin/munin09:10:163
2211799181,16cyclictest87350irq/126-enp1s0-10:30:163
22117991810,5cyclictest0-21swapper/312:35:233
2211199188,3cyclictest0-21swapper/209:21:582
2211199185,3cyclictest0-21swapper/207:52:592
2211199183,2cyclictest87350irq/126-enp1s0-09:08:302
2211199182,2cyclictest87450irq/127-enp1s0-07:29:512
2211199181,8cyclictest0-21swapper/211:42:072
2211199181,5cyclictest0-21swapper/208:43:582
22111991813,3cyclictest23489-21grep09:35:262
22111991813,3cyclictest10808-21diskmemload09:11:292
22111991812,3cyclictest27805-21fschecks_count07:20:142
22111991811,4cyclictest4705-21fgrep11:20:232
2211199181,14cyclictest0-21swapper/208:15:012
2211199180,6cyclictest0-21swapper/208:50:242
2210599182,4cyclictest6817-21irqstats07:45:171
2210599181,16cyclictest0-21swapper/108:59:141
2210199188,2cyclictest87350irq/126-enp1s0-10:40:120
2210199188,2cyclictest18020-21cat11:50:230
2210199185,4cyclictest2569-21df_inode08:50:130
2210199183,9cyclictest7-21ksoftirqd/008:15:010
2210199183,3cyclictest89050irq/133-enp2s0-07:15:130
2210199182,7cyclictest7869-21/usr/sbin/munin11:30:230
2210199181,15cyclictest0-21swapper/008:24:210
2210199181,15cyclictest0-21swapper/007:26:130
150472181,2sleep00-21swapper/009:18:140
2211799171,6cyclictest0-21swapper/310:41:363
2211799171,5cyclictest29572-21chrt12:19:243
2211799171,5cyclictest12609-21df_inode11:40:143
2211799171,4cyclictest7691-21/usr/sbin/munin09:00:213
2211799170,9cyclictest0-21swapper/307:20:263
2211799170,8cyclictest0-21swapper/309:16:283
2211199172,2cyclictest87450irq/127-enp1s0-07:33:552
2211199171,6cyclictest0-21swapper/210:05:182
2211199171,3cyclictest0-21swapper/210:52:482
22111991711,3cyclictest18980-21cpuspeed_turbos11:55:132
2211199171,13cyclictest0-21swapper/211:35:172
2211199171,10cyclictest8564-1kworker/u9:110:21:382
2210199172,7cyclictest27385-21/usr/sbin/munin07:20:180
2210199171,6cyclictest28975-21sshd11:04:380
2210199171,13cyclictest2953-21cpuspeed_turbos10:05:130
22101991711,3cyclictest27997-21/usr/sbin/munin08:35:210
2210199171,11cyclictest0-21swapper/010:28:070
2211199161,6cyclictest30495-21mailstats11:05:232
2210199164,3cyclictest2877-21smart_sda07:35:260
2210199162,11cyclictest31185-21grep08:40:180
22111991510,3cyclictest827-21runrttasks08:46:552
2210199154,2cyclictest11-21rcuc/008:15:220
2210199154,2cyclictest11-21rcuc/008:15:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional