You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-06 - 08:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Mon Jul 06, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
155522238163,24sleep30-21swapper/319:08:343
154262236165,24sleep20-21swapper/219:06:562
153462202166,24sleep10-21swapper/119:05:541
154092201164,25sleep00-21swapper/019:06:420
15756997774,2cyclictest0-21swapper/222:38:022
207672500,1sleep30-21swapper/320:30:243
282082450,2sleep04460-21diskmemload00:30:120
1575299381,24cyclictest0-21swapper/123:19:351
1575299380,32cyclictest0-21swapper/122:33:591
1575299361,25cyclictest0-21swapper/100:15:331
1576299351,11cyclictest0-21swapper/323:28:193
1575299351,21cyclictest5756-21ntp_states21:10:211
1575299351,21cyclictest5756-21ntp_states21:10:211
1574499356,14cyclictest23023-21idleruntime-cro19:25:010
1574499355,14cyclictest20820-21munin-run19:20:010
1576299331,23cyclictest4460-21diskmemload23:42:433
1576299331,12cyclictest0-21swapper/321:38:393
1576299331,11cyclictest0-21swapper/322:09:353
1576299330,23cyclictest0-21swapper/300:18:113
15752993320,5cyclictest0-21swapper/123:09:231
1575299331,23cyclictest0-21swapper/119:37:111
1576299321,10cyclictest0-21swapper/322:20:033
1575299321,9cyclictest0-21swapper/120:56:511
1575299321,22cyclictest0-21swapper/122:14:131
1574499324,20cyclictest5351-21kworker/0:023:44:050
1576299311,13cyclictest0-21swapper/300:21:123
1575299311,23cyclictest0-21swapper/123:12:071
1575299311,11cyclictest0-21swapper/120:36:271
1575299310,24cyclictest0-21swapper/123:53:451
15744993118,4cyclictest0-21swapper/023:52:580
1576299301,12cyclictest0-21swapper/321:54:513
1576299300,17cyclictest0-21swapper/322:15:033
1575299301,20cyclictest0-21swapper/119:26:431
1575299301,18cyclictest0-21swapper/120:28:351
1575299301,17cyclictest0-21swapper/120:20:231
1575299301,15cyclictest0-21swapper/119:58:591
1575299301,12cyclictest0-21swapper/123:25:351
1575299301,10cyclictest0-21swapper/119:10:451
1575299300,18cyclictest0-21swapper/121:44:191
1575299300,12cyclictest0-21swapper/123:47:111
1576299291,9cyclictest0-21swapper/323:46:273
15752992914,6cyclictest0-21swapper/100:25:311
1575299291,18cyclictest0-21swapper/119:47:271
1575299291,17cyclictest0-21swapper/119:40:431
1575299290,19cyclictest0-21swapper/121:15:231
1575299290,17cyclictest0-21swapper/123:20:111
1575299290,17cyclictest0-21swapper/121:20:231
1575299290,11cyclictest0-21swapper/121:47:031
1574499294,12cyclictest8867-21basename21:20:010
1574499293,13cyclictest14842-21idleruntime-cro20:20:010
1574499292,10cyclictest29166-21kworker/0:123:35:110
1574499292,10cyclictest29166-21kworker/0:123:35:110
1574499291,13cyclictest17925-21munin-run19:15:010
1576299289,3cyclictest0-21swapper/319:50:453
1576299281,18cyclictest0-21swapper/321:03:553
1576299280,9cyclictest0-21swapper/322:30:513
1576299280,10cyclictest0-21swapper/322:44:313
1576299280,10cyclictest0-21swapper/319:55:033
1575299289,9cyclictest0-21swapper/100:31:291
1575299289,4cyclictest27344-21hddtemp_smartct22:00:171
1575299281,8cyclictest0-21swapper/122:49:591
1575299281,3cyclictest0-21swapper/120:15:231
1575299281,21cyclictest0-21swapper/100:20:031
1575299281,17cyclictest0-21swapper/119:19:271
1575299281,14cyclictest0-21swapper/122:22:031
1575299281,10cyclictest0-21swapper/119:21:411
1575299280,9cyclictest0-21swapper/120:34:431
1575299280,20cyclictest0-21swapper/122:26:391
1575299280,20cyclictest0-21swapper/122:26:391
1575299280,17cyclictest0-21swapper/123:42:551
1575299280,10cyclictest0-21swapper/121:57:591
1576299275,3cyclictest0-21swapper/320:59:513
1576299271,10cyclictest0-21swapper/323:35:033
1576299271,10cyclictest0-21swapper/323:35:033
1576299271,10cyclictest0-21swapper/319:33:273
1576299271,10cyclictest0-21swapper/319:16:313
1576299270,9cyclictest0-21swapper/322:12:593
1576299270,12cyclictest0-21swapper/322:27:273
1576299270,12cyclictest0-21swapper/322:27:273
1576299270,12cyclictest0-21swapper/300:39:313
1575299277,13cyclictest0-21swapper/100:08:091
1575299271,9cyclictest0-21swapper/122:18:031
1575299271,9cyclictest0-21swapper/119:50:471
1575299271,11cyclictest0-21swapper/119:31:091
1575299271,10cyclictest0-21swapper/100:37:531
15752992710,9cyclictest0-21swapper/121:50:491
1575299270,7cyclictest0-21swapper/120:50:551
1575299270,12cyclictest0-21swapper/122:42:311
1575299270,11cyclictest0-21swapper/122:08:311
15744992718,2cyclictest0-21swapper/022:55:160
1576299269,4cyclictest0-21swapper/322:35:163
1576299266,3cyclictest0-21swapper/320:21:053
1576299266,3cyclictest0-21swapper/319:45:573
1576299261,9cyclictest0-21swapper/321:09:113
1576299261,9cyclictest0-21swapper/320:13:073
1576299261,8cyclictest0-21swapper/322:53:413
1576299261,11cyclictest0-21swapper/319:14:073
1576299261,10cyclictest0-21swapper/320:02:073
1576299261,10cyclictest0-21swapper/320:02:073
1576299260,9cyclictest0-21swapper/300:10:113
1576299260,11cyclictest0-21swapper/320:09:553
1576299260,10cyclictest0-21swapper/321:45:513
1575299267,9cyclictest0-21swapper/120:46:171
1575299261,9cyclictest0-21swapper/100:01:491
15752992611,7cyclictest0-21swapper/121:00:531
1575299261,16cyclictest0-21swapper/121:27:211
1575299261,16cyclictest0-21swapper/120:13:311
1575299261,12cyclictest21277-21fw_packets23:00:151
1575299261,12cyclictest0-21swapper/120:01:131
1575299261,12cyclictest0-21swapper/120:01:131
1575299261,11cyclictest0-21swapper/122:38:111
1576299255,3cyclictest0-21swapper/320:48:293
1576299252,20cyclictest28245-21/usr/sbin/munin20:50:203
1576299251,9cyclictest0-21swapper/300:02:233
1576299251,10cyclictest0-21swapper/300:31:153
1576299250,11cyclictest0-21swapper/323:21:193
1575699252,8cyclictest808-21snmpd23:15:192
1575299251,9cyclictest0-21swapper/123:55:231
1575299251,9cyclictest0-21swapper/120:08:391
1575299251,10cyclictest0-21swapper/120:43:531
1575299250,9cyclictest0-21swapper/122:55:071
1575299250,13cyclictest0-21swapper/100:14:251
1575299250,12cyclictest0-21swapper/121:38:511
1575299250,10cyclictest0-21swapper/122:50:071
1576299248,3cyclictest0-21swapper/323:56:253
1576299246,2cyclictest0-21swapper/321:41:393
1576299242,11cyclictest25732-21/usr/sbin/munin00:25:253
1576299241,20cyclictest4683-21/usr/sbin/munin21:10:233
1576299241,20cyclictest4683-21/usr/sbin/munin21:10:233
1576299241,20cyclictest20693-21/usr/sbin/munin23:00:183
1576299241,14cyclictest0-21swapper/319:42:473
1576299241,12cyclictest839-21lldpd22:45:463
1576299241,12cyclictest22898-21/usr/sbin/munin23:05:163
1576299241,11cyclictest18496-21/usr/sbin/munin22:55:213
1576299241,11cyclictest1506-21/usr/sbin/munin23:30:173
1576299240,10cyclictest0-21swapper/321:59:033
1575699244,6cyclictest23237-40dnf19:25:142
1575299241,11cyclictest0-21swapper/123:35:571
1575299241,11cyclictest0-21swapper/123:35:571
1575299241,10cyclictest0-21swapper/121:07:591
1575299240,10cyclictest0-21swapper/123:30:391
1574499249,14cyclictest0-21swapper/021:00:010
1574499241,15cyclictest25177-21idleruntime21:55:160
1576299237,3cyclictest0-21swapper/300:09:513
1576299232,7cyclictest23237-40dnf19:25:153
1576299231,13cyclictest0-21swapper/319:23:033
15762992310,3cyclictest0-21swapper/321:30:233
1576299230,9cyclictest0-21swapper/320:18:553
1575699231,4cyclictest0-21swapper/223:52:582
1575699230,5cyclictest18489-21kworker/u8:122:55:162
1575299230,10cyclictest0-21swapper/121:30:031
15744992318,4cyclictest87350irq/126-enp1s0-22:31:200
1574499231,15cyclictest0-21swapper/023:28:210
1574499230,5cyclictest0-21swapper/020:30:190
66242220,2sleep20-21swapper/222:25:252
66242220,2sleep20-21swapper/222:25:252
1576299228,2cyclictest0-21swapper/323:52:513
1576299222,16cyclictest808-21snmpd22:00:193
1576299221,11cyclictest18176-21memory20:25:193
1576299220,9cyclictest0-21swapper/320:39:193
1575699229,3cyclictest87650irq/129-enp1s0-23:20:562
1575699226,4cyclictest4570-21unixbench-2d22:20:262
1574499229,5cyclictest0-21swapper/000:37:060
1574499223,10cyclictest14715-21/usr/sbin/munin00:00:130
1574499221,4cyclictest11660-21cut20:10:200
1574499221,15cyclictest0-21swapper/021:35:420
1574499221,15cyclictest0-21swapper/000:26:200
1574499221,14cyclictest0-21swapper/023:34:410
1574499221,14cyclictest0-21swapper/022:26:070
1574499221,14cyclictest0-21swapper/022:26:070
1574499221,14cyclictest0-21swapper/022:01:440
1574499221,12cyclictest4460-21diskmemload22:10:180
1574499221,11cyclictest87350irq/126-enp1s0-22:39:410
1574499220,15cyclictest0-21swapper/021:11:270
1574499220,15cyclictest0-21swapper/021:11:270
1576299217,3cyclictest0-21swapper/321:15:373
1576299211,9cyclictest0-21swapper/319:36:373
15762992115,3cyclictest11428-21tr21:25:123
1576299211,11cyclictest0-21swapper/323:11:313
1576299211,10cyclictest0-21swapper/323:16:193
1576299211,10cyclictest0-21swapper/320:40:513
1575699215,3cyclictest2490-21missed_timers23:30:192
1575699213,5cyclictest15441-21latency_hist21:35:032
1574499212,12cyclictest18489-21kworker/u8:123:02:250
15744992116,3cyclictest4460-21diskmemload00:23:450
1574499211,15cyclictest0-21swapper/023:18:220
15744992110,3cyclictest0-21swapper/022:19:420
198822200,3sleep011-21rcuc/000:10:180
1576299200,10cyclictest0-21swapper/321:20:113
1575699209,2cyclictest15084-21needreboot22:45:192
1575699206,4cyclictest1407-21pmu-power21:00:232
1575699202,6cyclictest17651-21munin-run21:40:012
1575699202,5cyclictest580-21sh22:15:012
1575699201,8cyclictest0-21swapper/223:10:012
1575699201,5cyclictest0-21swapper/220:55:232
1575699201,4cyclictest27962-21sed22:00:252
1575699200,4cyclictest0-21swapper/223:00:162
1574499209,3cyclictest22270-21/usr/sbin/munin21:50:180
1574499208,2cyclictest20189-21kworker/0:123:10:230
1574499204,8cyclictest24001-21if_enp2s019:25:160
1574499203,3cyclictest1255-21head21:00:200
1574499203,10cyclictest14505-21kworker/u8:000:07:440
1574499201,8cyclictest0-21swapper/023:06:030
1574499201,7cyclictest27403-21turbostat.cron19:35:010
1574499201,14cyclictest0-21swapper/022:40:290
1574499201,14cyclictest0-21swapper/021:47:560
1574499201,14cyclictest0-21swapper/000:17:540
1574499201,11cyclictest9079-21/usr/sbin/munin21:20:150
1574499201,11cyclictest1870-21/usr/sbin/munin19:50:230
1574499201,11cyclictest1464-21sshd19:48:110
91332190,3sleep20-21swapper/222:30:482
1575699195,4cyclictest4460-21diskmemload00:00:172
1575699194,4cyclictest25439-21/usr/sbin/munin19:30:132
1575699192,6cyclictest30850-21munin-run22:10:012
1575699191,6cyclictest0-21swapper/221:20:082
1575699191,5cyclictest12229-21df22:40:132
1575699191,4cyclictest22634-21switchtime00:15:242
15756991913,3cyclictest17767-21grep22:50:262
15756991912,4cyclictest3136-21head21:05:162
1575699191,10cyclictest0-21swapper/223:40:502
1574499196,3cyclictest0-21swapper/022:05:180
1574499192,7cyclictest87350irq/126-enp1s0-21:30:080
1574499192,13cyclictest17214-21memory22:50:190
1574499191,9cyclictest0-21swapper/023:45:430
1574499191,9cyclictest0-21swapper/021:41:490
1574499191,6cyclictest11278-21/usr/sbin/munin21:25:240
1574499191,6cyclictest0-21swapper/022:20:480
1574499191,10cyclictest3477-21strings21:05:190
1574499191,10cyclictest29495-21/usr/sbin/munin23:20:280
1574499190,6cyclictest16057-21needreboot20:20:210
1574499190,17cyclictest0-21swapper/022:46:120
142842191,2sleep00-21swapper/023:57:390
1575699187,3cyclictest3841-21awk19:55:012
1575699181,7cyclictest18037-21latency_hist19:15:012
1575699181,6cyclictest29099-21ps00:30:222
1575699181,5cyclictest31686-21/usr/sbin/munin23:25:192
1575699181,5cyclictest12859-21/usr/sbin/munin20:15:152
1575699181,14cyclictest23196-21grep20:35:262
15756991811,4cyclictest12450-21proc_pri21:25:232
15756991811,4cyclictest10429-21idleruntime-cro20:10:012
1574499181,8cyclictest23852-21/usr/sbin/munin20:40:170
15744991811,4cyclictest9867-21/usr/sbin/munin20:05:240
1574499181,14cyclictest21657-21/usr/sbin/munin20:35:250
78762171,2sleep20-21swapper/220:00:392
78762171,2sleep20-21swapper/220:00:392
1575699176,3cyclictest87650irq/129-enp1s0-21:15:122
1575699175,4cyclictest30-21rcuc/221:10:152
1575699175,4cyclictest30-21rcuc/221:10:152
1575699174,2cyclictest87650irq/129-enp1s0-19:20:172
1575699173,5cyclictest16934-21/usr/sbin/munin00:05:192
1575699172,5cyclictest25001-21head20:40:222
1575699171,8cyclictest0-21swapper/200:21:582
1575699171,7cyclictest0-21swapper/222:16:372
1575699171,6cyclictest0-21swapper/223:55:212
1575699171,6cyclictest0-21swapper/221:50:042
1575699171,6cyclictest0-21swapper/220:47:212
1575699171,6cyclictest0-21swapper/200:35:342
1575699171,6cyclictest0-21swapper/200:11:112
15756991712,3cyclictest21002-21sort21:45:202
15756991712,3cyclictest11010-21df_inode20:10:152
1575699170,7cyclictest25461-21needreboot21:55:192
1574499171,9cyclictest0-21swapper/019:35:060
1574499171,4cyclictest808-21snmpd20:25:130
1574499171,4cyclictest28358-21grep20:50:130
40602160,2sleep00-21swapper/019:55:110
1575699167,3cyclictest0-21swapper/200:27:062
1575699166,7cyclictest8256-21cpuspeed_turbos23:45:122
1575699164,3cyclictest19172-21switchtime21:40:252
1575699163,4cyclictest0-21swapper/220:25:162
1575699162,7cyclictest0-21swapper/220:22:502
1575699161,6cyclictest32214-21ls19:45:112
15756991612,3cyclictest0-21swapper/219:56:042
15756991611,3cyclictest27638-21/usr/sbin/munin19:35:262
15756991611,3cyclictest25098-21acpi23:10:132
1575699161,10cyclictest0-21swapper/219:41:062
15756991610,3cyclictest18179-21fschecks_count19:15:022
1575699160,4cyclictest0-21swapper/223:35:132
1575699160,4cyclictest0-21swapper/223:35:132
1574499161,3cyclictest87550irq/128-enp1s0-20:00:180
1574499161,3cyclictest87550irq/128-enp1s0-20:00:180
15744991610,4cyclictest27716-21sshd20:46:160
1575699151,7cyclictest0-21swapper/220:50:562
15756991510,3cyclictest20525-21sh20:30:212
1574499152,2cyclictest31393-21sed19:40:270
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional