You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-10 - 14:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Fri Jul 10, 2026 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5532237164,24sleep30-21swapper/307:08:393
5162222192,20sleep20-21swapper/207:08:122
3172204166,26sleep00-21swapper/007:05:460
4222202168,22sleep10-21swapper/107:07:031
2931021520,4sleep281199cyclictest08:10:242
811996663,2cyclictest0-21swapper/207:19:522
811995350,2cyclictest0-21swapper/210:43:182
93902500,2sleep30-21swapper/307:25:243
221172490,3sleep011-21rcuc/011:35:180
81599442,27cyclictest5778-21/usr/sbin/munin11:00:213
81599378,15cyclictest3842-21sshd12:08:243
81599377,17cyclictest20351-21grep09:05:153
81599377,17cyclictest20351-21grep09:05:143
81599371,6cyclictest22011-21chrt11:35:183
81599361,6cyclictest1022-21/usr/sbin/munin07:10:253
815993610,23cyclictest18975-21/usr/sbin/munin11:30:173
81199361,14cyclictest3863-21latency09:40:192
79299361,26cyclictest0-21swapper/110:40:031
78699368,13cyclictest4312-21idleruntime-cro08:30:020
81599354,3cyclictest12050irq/123-ahci[0012:00:023
81599351,7cyclictest25566-21/usr/sbin/munin11:45:243
81599349,14cyclictest87350irq/126-enp1s0-07:40:353
81599341,6cyclictest22704-21fschecks_time09:10:163
81199341,12cyclictest0-21swapper/212:05:152
307332340,4sleep078699cyclictest10:40:250
81599333,3cyclictest87650irq/129-enp1s0-09:30:203
81599331,7cyclictest28109-21diskstats11:50:143
815993310,20cyclictest9262-21switchtime11:05:233
815993310,20cyclictest6335-21processes09:45:223
815993310,20cyclictest11752-21/usr/sbin/munin10:00:243
811993320,8cyclictest0-21swapper/212:04:352
81199331,30cyclictest0-21swapper/212:35:062
79299331,23cyclictest0-21swapper/107:47:391
79299331,18cyclictest0-21swapper/109:08:011
79299331,18cyclictest0-21swapper/109:08:011
81599321,6cyclictest21909-21diskmemload11:28:173
81599321,6cyclictest21909-21diskmemload11:28:173
81599321,6cyclictest0-21swapper/309:42:443
81599321,5cyclictest31222-21memory08:15:213
815993210,19cyclictest21580-21ntp_kernel_err10:20:203
811993218,8cyclictest0-21swapper/209:10:352
81199321,14cyclictest0-21swapper/210:58:522
79299320,24cyclictest0-21swapper/109:46:101
81599319,19cyclictest4549-21/usr/sbin/munin08:30:213
81599319,19cyclictest32503-21/usr/sbin/munin08:20:183
81599319,18cyclictest16013-21timerwakeupswit11:20:243
81599318,20cyclictest827-21runrttasks10:14:113
81599318,20cyclictest0-21swapper/307:55:283
81599312,4cyclictest2353-21/usr/sbin/munin08:25:313
81599311,6cyclictest5189-21kworker/u8:211:10:203
81599311,5cyclictest0-21swapper/310:19:563
815993110,18cyclictest26104-21ls08:05:143
815993110,18cyclictest24087-21/usr/sbin/munin10:25:243
81199311,19cyclictest0-21swapper/211:15:052
79299310,12cyclictest0-21swapper/109:10:131
78699316,12cyclictest18752-21munin-run11:30:010
78699316,12cyclictest18752-21munin-run11:30:000
78699315,13cyclictest7136-21munin-run09:50:010
81599309,18cyclictest29338-21/usr/sbin/munin10:40:203
81599308,19cyclictest12119-21vmstat07:30:293
81599303,3cyclictest87650irq/129-enp1s0-09:15:253
81599302,3cyclictest11813-21hddtemp_smartct08:45:163
81599301,6cyclictest14018-21grep10:05:123
81599301,5cyclictest13670-21df08:50:123
81599301,4cyclictest913-21/usr/sbin/munin09:35:133
81599301,4cyclictest24059-21/usr/sbin/munin08:00:143
81599301,4cyclictest23551-21/usr/sbin/munin11:40:133
81599301,3cyclictest11849-21tr12:25:233
81599301,19cyclictest14545-21grep12:30:253
811993022,3cyclictest0-21swapper/211:42:272
81199301,13cyclictest0-21swapper/212:22:332
81199301,13cyclictest0-21swapper/209:58:412
81199301,12cyclictest0-21swapper/210:23:522
79299301,3cyclictest21909-21diskmemload10:39:091
79299301,18cyclictest0-21swapper/112:20:121
79299301,13cyclictest0-21swapper/111:00:331
79299300,18cyclictest0-21swapper/111:10:101
81599299,17cyclictest1998-21grep10:50:153
81599294,3cyclictest7697-21/usr/sbin/munin09:50:143
81599292,5cyclictest20575-21smart_sda07:50:243
81599292,4cyclictest9862-21smart_sda12:20:243
81599292,4cyclictest9287-21diskstats08:40:143
81599292,4cyclictest29794-21ntp_states09:25:243
81599292,4cyclictest25626-21hddtemp_smartct10:30:163
81599292,4cyclictest15541-21/usr/sbin/munin12:35:143
81599292,3cyclictest13064-21df_inode07:35:143
81599291,6cyclictest0-21swapper/311:16:163
81599291,5cyclictest7475-21ntp_states12:15:223
81599291,5cyclictest32216-21egrep12:00:113
81599291,4cyclictest29378-21smart_sda08:10:273
81599291,4cyclictest16588-21/usr/sbin/munin08:55:213
81599291,3cyclictest18118-21/usr/sbin/munin07:45:223
81599291,24cyclictest8227-21cat07:25:023
815992912,14cyclictest4360-21expr12:10:123
815992910,16cyclictest827-21runrttasks08:38:473
81199291,12cyclictest0-21swapper/211:13:262
81199291,12cyclictest0-21swapper/209:31:132
81199291,11cyclictest0-21swapper/210:05:202
79299291,25cyclictest0-21swapper/107:30:511
79299291,12cyclictest0-21swapper/111:48:051
79299291,11cyclictest0-21swapper/109:39:121
79299290,22cyclictest0-21swapper/111:29:131
79299290,22cyclictest0-21swapper/111:29:131
78699296,11cyclictest19103-21perf07:50:010
81599287,18cyclictest1142-21/usr/sbin/munin10:50:023
81599282,4cyclictest27484-21df_inode10:35:133
81599281,4cyclictest4264-21/usr/sbin/munin10:55:173
81599281,4cyclictest10579-21/usr/sbin/munin09:55:203
81599281,3cyclictest87650irq/129-enp1s0-09:20:223
81599281,3cyclictest3758-21hddtemp_smartct07:15:043
81199281,19cyclictest0-21swapper/210:02:172
81199281,18cyclictest0-21swapper/208:40:182
81199281,13cyclictest0-21swapper/209:19:012
81199281,12cyclictest0-21swapper/209:52:332
79299281,9cyclictest0-21swapper/108:53:491
79299281,11cyclictest0-21swapper/111:58:211
79299281,11cyclictest0-21swapper/109:52:081
79299281,11cyclictest0-21swapper/108:57:091
79299281,10cyclictest0-21swapper/109:01:051
79299281,10cyclictest0-21swapper/108:41:551
79299281,10cyclictest0-21swapper/108:13:451
79299280,17cyclictest0-21swapper/109:22:331
79299280,17cyclictest0-21swapper/107:55:101
79299280,13cyclictest0-21swapper/107:50:271
79299280,12cyclictest0-21swapper/110:12:361
79299280,10cyclictest0-21swapper/109:43:401
81599271,4cyclictest18833-21/usr/sbin/munin09:00:233
811992715,6cyclictest0-21swapper/210:29:062
811992712,3cyclictest26878-21diskstats09:20:152
81199271,13cyclictest0-21swapper/210:12:002
81199271,13cyclictest0-21swapper/208:00:262
81199271,12cyclictest0-21swapper/212:16:292
81199271,12cyclictest0-21swapper/208:29:082
81199271,11cyclictest0-21swapper/211:02:462
79299277,8cyclictest0-21swapper/111:32:311
79299271,8cyclictest0-21swapper/110:52:211
79299271,8cyclictest0-21swapper/107:14:041
79299271,17cyclictest0-21swapper/107:36:141
79299271,16cyclictest0-21swapper/110:46:191
79299271,10cyclictest0-21swapper/107:42:401
79299270,14cyclictest0-21swapper/110:27:081
79299270,11cyclictest0-21swapper/111:54:531
79299270,11cyclictest0-21swapper/111:39:531
79299270,11cyclictest0-21swapper/110:55:131
78699272,12cyclictest21926-21basename09:10:000
78699272,12cyclictest21926-21basename09:10:000
165852270,2sleep00-21swapper/008:55:220
81199261,3cyclictest0-21swapper/210:53:302
811992613,4cyclictest0-21swapper/209:27:352
81199261,12cyclictest0-21swapper/211:29:332
81199261,12cyclictest0-21swapper/211:29:332
81199261,12cyclictest0-21swapper/210:19:222
81199261,12cyclictest0-21swapper/208:35:122
81199261,11cyclictest0-21swapper/208:48:252
79299261,9cyclictest0-21swapper/111:21:451
79299261,9cyclictest0-21swapper/110:04:441
79299261,9cyclictest0-21swapper/109:26:561
79299261,16cyclictest0-21swapper/112:38:541
79299261,13cyclictest0-21swapper/112:10:561
79299261,11cyclictest0-21swapper/108:31:291
79299261,10cyclictest0-21swapper/112:02:171
79299261,10cyclictest0-21swapper/109:32:241
79299261,10cyclictest0-21swapper/109:15:331
79299261,10cyclictest0-21swapper/108:00:351
79299260,11cyclictest0-21swapper/112:29:561
79299260,11cyclictest0-21swapper/112:17:201
79299260,11cyclictest0-21swapper/110:05:321
78699267,10cyclictest13175-21cstates12:30:110
786992610,14cyclictest13120-21perf08:50:010
81199251,9cyclictest13401-21smartctl07:35:162
811992518,2cyclictest0-21swapper/210:31:582
811992517,3cyclictest0-21swapper/207:32:432
811992514,3cyclictest0-21swapper/211:05:302
811992514,3cyclictest0-21swapper/209:47:312
81199251,14cyclictest0-21swapper/210:39:082
81199251,11cyclictest0-21swapper/211:52:432
81199251,11cyclictest0-21swapper/208:54:472
81199251,10cyclictest0-21swapper/207:10:112
81199250,11cyclictest0-21swapper/207:56:472
79299256,8cyclictest0-21swapper/111:07:271
79299251,8cyclictest0-21swapper/109:57:561
792992511,3cyclictest0-21swapper/108:18:551
79299251,10cyclictest0-21swapper/112:05:561
79299250,17cyclictest0-21swapper/108:20:271
79299250,16cyclictest0-21swapper/111:44:111
78699258,14cyclictest21300-21idleruntime-cro07:55:010
78699257,15cyclictest11530-21munin-run10:00:010
78699251,13cyclictest3370-21basename10:55:020
81199241,13cyclictest0-21swapper/210:48:502
81199241,12cyclictest0-21swapper/211:55:492
81199241,11cyclictest0-21swapper/207:46:572
79299247,7cyclictest0-21swapper/107:26:581
79299241,10cyclictest0-21swapper/108:28:211
79299240,10cyclictest0-21swapper/108:45:051
78699242,10cyclictest29129-21perf10:40:020
81199239,3cyclictest0-21swapper/211:46:152
81199237,3cyclictest0-21swapper/208:23:582
81199236,3cyclictest0-21swapper/212:29:032
81199234,2cyclictest0-21swapper/207:50:192
811992313,3cyclictest0-21swapper/208:17:062
81199231,11cyclictest0-21swapper/208:34:022
81199230,12cyclictest0-21swapper/211:32:072
79299238,7cyclictest0-21swapper/108:39:311
79299235,12cyclictest0-21swapper/111:15:111
79299231,10cyclictest0-21swapper/110:22:381
79299230,9cyclictest0-21swapper/110:31:571
79299230,9cyclictest0-21swapper/110:18:001
79299230,7cyclictest0-21swapper/107:20:571
79299230,10cyclictest0-21swapper/112:31:041
78699239,11cyclictest26273-21sort11:45:150
78699236,7cyclictest0-21swapper/010:18:060
81199229,3cyclictest0-21swapper/211:35:312
81199229,3cyclictest0-21swapper/211:21:172
81199228,3cyclictest0-21swapper/212:32:272
81199225,3cyclictest0-21swapper/209:07:392
81199225,3cyclictest0-21swapper/209:07:392
811992211,7cyclictest0-21swapper/209:38:392
79299221,10cyclictest0-21swapper/107:17:161
78699226,6cyclictest0-21swapper/011:15:020
78699226,5cyclictest23380-21kworker/0:007:59:160
78699223,9cyclictest2061050kworker/u8:210:30:040
81199211,14cyclictest0-21swapper/212:10:132
81199210,11cyclictest0-21swapper/207:40:122
79299211,10cyclictest0-21swapper/108:06:181
78699213,3cyclictest0-21swapper/012:35:080
78699213,11cyclictest11359-21ps11:10:230
78699212,10cyclictest14580-21/usr/sbin/munin11:20:130
78699211,4cyclictest6850-21sh11:00:210
78699211,13cyclictest23261-21cut09:10:230
78699211,12cyclictest30307-21/usr/sbin/munin08:15:290
78699211,11cyclictest0-21swapper/011:54:330
81199201,13cyclictest0-21swapper/207:26:042
81199201,12cyclictest0-21swapper/207:24:522
81199201,11cyclictest0-21swapper/208:09:262
78699209,3cyclictest21270-21basename10:20:160
78699209,3cyclictest10800-21/usr/sbin/munin12:25:150
78699206,4cyclictest0-21swapper/012:12:280
78699205,3cyclictest30072-21/usr/sbin/munin09:25:260
78699203,3cyclictest0-21swapper/011:40:060
78699202,10cyclictest7986-21/usr/sbin/munin11:05:140
78699201,3cyclictest7720-21users07:20:260
78699201,13cyclictest0-21swapper/011:57:450
78699201,11cyclictest8273-21memory09:50:200
78699201,11cyclictest21909-21diskmemload10:06:010
78699201,11cyclictest13333-21/usr/sbin/munin08:50:170
78699201,11cyclictest10512-21sshd12:24:240
78699201,11cyclictest0-21swapper/010:00:120
78699201,11cyclictest0-21swapper/009:20:270
78699201,10cyclictest808-21snmpd07:17:080
78699201,10cyclictest2604-21sed07:10:270
78699200,9cyclictest0-21swapper/012:05:050
81199193,3cyclictest0-21swapper/209:03:192
78699198,4cyclictest0-21swapper/010:45:030
78699192,14cyclictest671-21tail08:20:170
78699191,15cyclictest0-21swapper/008:08:280
78699191,11cyclictest0-21swapper/010:55:270
78699191,11cyclictest0-21swapper/010:26:200
78699191,11cyclictest0-21swapper/009:18:120
78699191,10cyclictest8938-21/usr/sbin/munin08:40:120
78699191,10cyclictest7489-21open_files12:15:210
78699191,10cyclictest5153-21grep08:30:180
78699191,10cyclictest0-21swapper/011:33:390
78699191,10cyclictest0-21swapper/009:30:330
78699185,8cyclictest0-21swapper/008:01:160
78699185,4cyclictest0-21swapper/010:11:010
78699181,7cyclictest0-21swapper/008:10:240
78699181,10cyclictest0-21swapper/009:35:140
78699180,10cyclictest0-21swapper/007:35:120
46332180,2sleep00-21swapper/009:41:180
81199171,13cyclictest0-21swapper/208:57:292
78699176,3cyclictest808-21snmpd07:32:100
78699173,3cyclictest0-21swapper/012:01:530
78699171,7cyclictest0-21swapper/009:03:010
78699171,4cyclictest8713-21munin-run08:40:010
78699170,9cyclictest11-21rcuc/007:25:250
78699164,4cyclictest14984-21cpuspeed_turbos07:40:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional