You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-31 - 19:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Sun May 31, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
169792239163,24sleep30-21swapper/307:09:563
168172227156,58sleep10-21swapper/107:07:521
166412203165,26sleep20-21swapper/207:05:362
168212202165,26sleep00-21swapper/007:07:550
765121620,3sleep01706899cyclictest10:25:200
1300221620,3sleep01706899cyclictest11:50:240
235312460,3sleep223532-21grep11:00:252
223812460,4sleep122380-21needreboot09:45:191
315692420,1sleep10-21swapper/111:20:151
315692420,1sleep10-21swapper/111:20:141
1706899388,14cyclictest2628-21idleruntime-cro11:30:010
122312380,4sleep339-21rcuc/311:50:153
17075993310,11cyclictest0-21swapper/109:10:231
1708699321,15cyclictest0-21swapper/310:47:583
1706899318,11cyclictest0-21swapper/009:10:010
17086993017,7cyclictest0-21swapper/312:36:583
17086993017,7cyclictest0-21swapper/312:36:583
1708699301,18cyclictest0-21swapper/308:15:183
1708699299,9cyclictest0-21swapper/308:08:553
1708699291,12cyclictest0-21swapper/310:57:263
1708699291,12cyclictest0-21swapper/309:40:193
1708099291,6cyclictest21159-21idleruntime12:10:162
1707599291,25cyclictest0-21swapper/112:10:161
1707599291,20cyclictest15128-21taskset10:44:281
1708699281,12cyclictest0-21swapper/312:01:233
1708699281,12cyclictest0-21swapper/311:12:183
1708699280,11cyclictest0-21swapper/307:40:073
1707599284,3cyclictest3013-21kworker/1:107:48:151
1708699271,12cyclictest0-21swapper/312:24:583
1708699271,11cyclictest0-21swapper/311:58:393
1708699270,11cyclictest0-21swapper/308:10:273
1706899272,13cyclictest23378-21perf09:50:000
1706899272,12cyclictest0-21swapper/010:45:020
1708699269,3cyclictest0-21swapper/311:01:463
1708699267,10cyclictest0-21swapper/309:34:303
1708699263,2cyclictest30193-21/usr/sbin/munin10:05:273
1708699261,17cyclictest0-21swapper/309:50:353
1708699261,12cyclictest0-21swapper/311:48:193
1708699261,11cyclictest0-21swapper/311:25:153
1708699261,10cyclictest755-21rngd11:34:473
17086992610,3cyclictest0-21swapper/308:46:023
1707599261,17cyclictest0-21swapper/107:57:481
1707599261,15cyclictest17205-21kworker/u8:210:49:431
1706899264,11cyclictest22761-21awk08:35:010
1708099258,4cyclictest18725-21timerwakeupswit07:10:252
17080992511,4cyclictest29977-21if_enp2s012:30:152
1707599251,17cyclictest0-21swapper/108:40:141
1707599251,16cyclictest0-21swapper/110:30:171
1707599251,15cyclictest0-21swapper/110:27:381
1707599251,12cyclictest0-21swapper/109:25:291
1707599250,17cyclictest0-21swapper/110:22:231
1706899252,7cyclictest0-21swapper/011:08:350
1706899251,12cyclictest29004-21munin-run12:30:010
1708699249,3cyclictest32572-21irqstats08:55:183
1708699249,3cyclictest0-21swapper/310:35:463
17086992414,3cyclictest0-21swapper/309:45:033
1708699241,14cyclictest0-21swapper/312:30:183
1708699240,13cyclictest0-21swapper/310:30:023
1708099246,4cyclictest7985-21perf09:15:022
17075992419,3cyclictest15804-21awk12:00:011
1707599241,17cyclictest0-21swapper/107:32:421
1707599241,16cyclictest0-21swapper/112:21:521
1707599241,16cyclictest0-21swapper/107:27:301
1707599241,11cyclictest19394-21latency_hist07:15:011
1707599241,11cyclictest0-21swapper/112:08:011
1707599241,11cyclictest0-21swapper/111:42:221
1707599241,10cyclictest471-21unixbench_multi12:35:271
1707599241,10cyclictest471-21unixbench_multi12:35:271
1707599240,5cyclictest11211-21kworker/u8:112:02:551
1707599240,17cyclictest0-21swapper/112:32:151
1707599240,15cyclictest6676-21kworker/u8:209:18:591
1707599240,14cyclictest0-21swapper/109:51:021
1706899241,11cyclictest5774-21diskmemload10:50:020
41552231,18sleep14248-21tune2fs09:05:161
1708699236,3cyclictest0-21swapper/309:18:303
1708699236,3cyclictest0-21swapper/308:30:063
1708699234,9cyclictest23825-21cat08:35:183
17086992312,3cyclictest0-21swapper/309:06:263
1708699231,13cyclictest16614-21kworker/u8:107:15:123
1708099236,4cyclictest31539-21awk08:55:012
1708099232,19cyclictest0-21swapper/211:08:532
1708099231,7cyclictest10194-21if_enp2s011:45:162
17080992317,3cyclictest4245-21ntp_states07:50:202
17080992317,3cyclictest4245-21ntp_states07:50:202
1707599237,9cyclictest0-21swapper/107:50:171
1707599237,9cyclictest0-21swapper/107:50:171
1707599237,5cyclictest0-21swapper/111:03:221
1707599231,17cyclictest0-21swapper/108:29:541
1707599231,16cyclictest0-21swapper/107:24:571
1707599231,15cyclictest0-21swapper/108:07:191
1707599231,13cyclictest23703-21ntpq12:15:211
1707599231,11cyclictest27181-21if_enp2s011:10:171
1707599231,10cyclictest0-21swapper/111:29:161
1707599230,21cyclictest0-21swapper/110:09:091
1707599230,17cyclictest11211-21kworker/u8:111:49:131
17068992317,3cyclictest3886-21idleruntime07:50:160
17068992317,3cyclictest3886-21idleruntime07:50:160
1708699228,3cyclictest0-21swapper/307:55:233
1708699226,3cyclictest0-21swapper/307:51:553
1708699226,3cyclictest0-21swapper/307:51:553
1708699225,7cyclictest181252sleep308:22:143
1708699224,2cyclictest0-21swapper/308:40:183
1708699221,7cyclictest0-21swapper/312:16:123
17086992211,3cyclictest0-21swapper/310:41:383
1708699221,13cyclictest0-21swapper/310:20:233
1708699221,11cyclictest0-21swapper/309:20:103
17086992210,4cyclictest0-21swapper/311:05:223
1708699220,9cyclictest0-21swapper/311:23:223
1708699220,9cyclictest0-21swapper/311:23:223
1708099229,4cyclictest16155-21awk08:20:012
1708099221,6cyclictest0-21swapper/210:09:092
1707599225,4cyclictest28604-21grep10:00:151
1707599225,14cyclictest0-21swapper/111:10:021
17075992218,3cyclictest0-21swapper/109:20:241
17075992217,3cyclictest0-21swapper/108:36:311
1707599221,6cyclictest12314-21switchtime10:35:241
1707599221,19cyclictest0-21swapper/108:15:161
1707599221,15cyclictest17205-21kworker/u8:210:54:451
1707599221,15cyclictest0-21swapper/112:27:081
1707599221,12cyclictest0-21swapper/111:30:501
1707599221,11cyclictest0-21swapper/109:38:551
1707599221,10cyclictest0-21swapper/110:14:431
1707599221,10cyclictest0-21swapper/110:14:431
1707599220,6cyclictest0-21swapper/111:16:431
1707599220,16cyclictest0-21swapper/110:15:201
1707599220,16cyclictest0-21swapper/109:40:171
1706899223,16cyclictest19250-21munin-run07:15:020
1706899221,12cyclictest27296-21unixbench_multi09:55:250
69692211,16sleep27069-21fschecks_count10:25:142
1708699215,6cyclictest0-21swapper/310:14:043
1708699215,6cyclictest0-21swapper/310:14:043
1708699215,3cyclictest0-21swapper/310:01:033
1708099216,4cyclictest0-21swapper/210:24:202
1708099211,8cyclictest0-21swapper/207:20:112
1708099211,7cyclictest1-21systemd11:30:022
1708099211,7cyclictest0-21swapper/212:05:282
1707599217,8cyclictest0-21swapper/107:18:201
17075992115,3cyclictest763-21dbus-daemon09:30:201
17075992115,3cyclictest16967-21grep08:20:151
1707599211,14cyclictest0-21swapper/110:55:391
17075992111,3cyclictest12305-21sh11:50:141
1707599211,11cyclictest8841-21smart_sda08:00:251
1707599211,11cyclictest5042-21/usr/sbin/munin11:35:171
1707599211,11cyclictest31480-21df07:40:131
1707599210,15cyclictest0-21swapper/109:58:021
1707599210,10cyclictest30119-21ps07:35:231
1706899214,8cyclictest7343-21sh08:00:010
1706899212,11cyclictest9435-21/usr/sbin/munin11:45:250
1706899211,6cyclictest0-21swapper/010:35:560
1706899211,4cyclictest2020-21basename10:15:010
1706899211,4cyclictest2020-21basename10:15:010
17068992111,3cyclictest87550irq/128-enp1s0-10:05:080
1708699209,3cyclictest17743-21cpuspeed_turbos10:50:133
1708699207,3cyclictest0-21swapper/307:36:033
1708699206,3cyclictest0-21swapper/309:14:383
1708699205,8cyclictest17338-21df_inode09:35:153
1708699204,3cyclictest0-21swapper/310:29:593
1708699202,10cyclictest0-21swapper/309:58:353
1708699201,5cyclictest0-21swapper/309:27:133
1708699201,10cyclictest0-21swapper/308:27:223
1708699200,13cyclictest0-21swapper/312:13:303
1708699200,13cyclictest0-21swapper/307:10:103
1708699200,11cyclictest29087-21fschecks_time11:15:143
1708099206,3cyclictest87350irq/126-enp1s0-08:10:162
1708099202,3cyclictest87650irq/129-enp1s0-11:21:332
1708099202,3cyclictest87650irq/129-enp1s0-11:21:332
1708099201,7cyclictest8203-21/usr/sbin/munin09:15:242
1708099201,7cyclictest27005-21rt-features09:55:232
1708099201,6cyclictest6640-21smart_sda07:55:242
1708099201,6cyclictest32606-21kernelversion08:55:182
1708099201,6cyclictest0-21swapper/210:36:342
1708099201,4cyclictest0-21swapper/211:56:192
1707599207,3cyclictest0-21swapper/108:30:221
17075992016,3cyclictest0-21swapper/108:14:141
1707599201,14cyclictest0-21swapper/109:02:141
1707599201,14cyclictest0-21swapper/109:02:141
1706899209,3cyclictest26811-21awk12:25:010
1706899201,6cyclictest0-21swapper/011:21:530
1706899201,6cyclictest0-21swapper/011:21:530
1706899201,6cyclictest0-21swapper/011:14:170
1706899201,11cyclictest20487-21switchtime09:40:230
1706899201,11cyclictest12597-21/usr/sbin/munin09:25:240
1706899201,10cyclictest23348-21grep07:20:170
1706899201,10cyclictest15799-21munin-run12:00:020
1708699199,2cyclictest7579-21/usr/sbin/munin08:00:163
1708699197,3cyclictest0-21swapper/307:29:263
1708699192,5cyclictest28294-21smart_sda12:25:243
1708699191,5cyclictest0-21swapper/310:18:293
1708699191,11cyclictest0-21swapper/312:06:323
1708699191,11cyclictest0-21swapper/311:35:173
1708699191,11cyclictest0-21swapper/307:23:223
1708699190,2cyclictest12050irq/123-ahci[0009:01:523
1708699190,2cyclictest12050irq/123-ahci[0009:01:523
1708099195,3cyclictest87350irq/126-enp1s0-07:50:022
1708099194,3cyclictest12050irq/123-ahci[0009:00:152
1708099194,3cyclictest12050irq/123-ahci[0009:00:142
1708099193,3cyclictest0-21swapper/208:35:242
1708099192,9cyclictest32-21ksoftirqd/212:20:272
1708099192,6cyclictest21447-21kworker/2:209:25:152
1708099191,6cyclictest22634-21/usr/sbin/munin12:15:152
1708099191,6cyclictest18789-21ps10:50:232
1708099191,6cyclictest0-21swapper/210:49:042
1708099191,5cyclictest26114-21sed07:25:262
1708099191,5cyclictest0-21swapper/207:18:422
17080991913,3cyclictest28381-21idleruntime-cro11:15:012
1708099191,15cyclictest26409-21wc08:40:232
1708099191,10cyclictest0-21swapper/210:56:002
1708099191,10cyclictest0-21swapper/210:40:202
1707599191,9cyclictest28218-21latency08:45:191
1707599191,10cyclictest1356-21munin-run09:00:021
1706899192,8cyclictest29574-21diskstats12:30:130
1706899191,9cyclictest11391-21grep09:20:200
1706899191,7cyclictest0-21swapper/011:30:210
17068991912,4cyclictest827-21runrttasks07:35:000
1706899191,11cyclictest4425-21/usr/sbin/munin10:20:230
1706899191,10cyclictest11983-21/usr/sbin/munin08:10:220
1706899191,10cyclictest0-21swapper/012:39:470
1706899191,10cyclictest0-21swapper/012:39:470
1706899191,10cyclictest0-21swapper/009:35:260
17068991910,6cyclictest16730-21systemd-journal09:34:440
28432181,2sleep00-21swapper/007:48:450
1708099185,3cyclictest87550irq/128-enp1s0-10:10:162
1708099185,3cyclictest87550irq/128-enp1s0-10:10:162
1708099184,3cyclictest87350irq/126-enp1s0-11:35:252
1708099184,3cyclictest24861-21swap09:50:242
1708099183,4cyclictest790-21in:imjournal11:51:352
1708099181,6cyclictest4137-21switchtime11:30:252
1708099181,6cyclictest28594-21gzip08:45:232
1708099181,6cyclictest0-21swapper/211:41:472
1708099181,4cyclictest32207-21if_err_enp2s012:35:192
1708099181,4cyclictest32207-21if_err_enp2s012:35:182
17080991813,3cyclictest10960-21processes08:05:232
1708099181,2cyclictest0-21swapper/209:30:122
1708099181,15cyclictest0-21swapper/207:34:332
1708099181,11cyclictest0-21swapper/209:45:012
1707599181,4cyclictest30375-21irqstats08:50:191
1706899184,2cyclictest87550irq/128-enp1s0-12:02:120
1706899181,9cyclictest31748-21tail07:40:160
1706899181,8cyclictest8853-21gltestperf09:15:150
1706899181,8cyclictest0-21swapper/010:02:090
1706899181,7cyclictest0-21swapper/012:19:440
1706899181,5cyclictest0-21swapper/010:17:050
1706899181,3cyclictest4755-21kworker/0:211:40:180
1706899181,14cyclictest6004-21/usr/sbin/munin09:10:230
1706899181,11cyclictest0-21swapper/011:18:540
143492181,2sleep00-21swapper/008:15:140
295632170,2sleep30-21swapper/308:50:123
199422170,2sleep00-21swapper/012:06:430
1708699176,3cyclictest0-21swapper/311:40:163
1708099174,3cyclictest87350irq/126-enp1s0-08:25:152
1708099173,3cyclictest11640-21fgrep09:20:242
1708099172,6cyclictest30-21rcuc/210:30:152
1708099172,4cyclictest29551-21gltestperf07:35:172
1708099172,3cyclictest0-21swapper/208:20:592
1708099172,14cyclictest0-21swapper/210:19:172
1708099172,14cyclictest0-21swapper/209:06:332
1708099171,9cyclictest0-21swapper/212:00:142
1708099171,8cyclictest0-21swapper/208:31:582
1708099171,7cyclictest0-21swapper/209:35:212
1708099171,5cyclictest28704-21idleruntime10:00:162
1708099171,5cyclictest27029-21/usr/sbin/munin12:25:252
1708099171,5cyclictest0-21swapper/209:49:302
1708099171,4cyclictest808-21snmpd11:17:562
17080991710,4cyclictest31616-21grep07:40:162
17080991710,4cyclictest0-21swapper/208:00:152
1706899177,6cyclictest0-21swapper/008:29:570
1706899177,3cyclictest87550irq/128-enp1s0-12:12:010
1706899174,6cyclictest87550irq/128-enp1s0-09:00:140
1706899174,6cyclictest87550irq/128-enp1s0-09:00:140
1706899171,9cyclictest0-21swapper/011:03:230
1706899171,7cyclictest0-21swapper/008:50:260
1706899171,5cyclictest7924-21df_inode08:00:150
1706899171,2cyclictest5774-21diskmemload10:58:510
17068991712,3cyclictest9500-21hwlatdetect10:30:170
17068991712,3cyclictest25141-21gltestperf07:25:150
17068991711,3cyclictest5209-21cstates11:35:130
1706899171,13cyclictest10773-21needreboot08:05:210
267942160,2sleep00-21swapper/008:40:430
1708699168,2cyclictest0-21swapper/307:45:193
17086991611,3cyclictest26755-21/usr/sbin/munin07:30:243
1706899163,3cyclictest28217-21latency08:45:180
1706899161,7cyclictest17618-21/usr/sbin/munin10:50:130
1706899161,5cyclictest24190-21tail09:50:140
1706899161,2cyclictest0-21swapper/007:15:030
17068991612,3cyclictest0-21swapper/008:35:500
17068991612,3cyclictest0-21swapper/008:21:380
17068991611,3cyclictest30410-21unixbench-2d07:35:260
1706899160,10cyclictest0-21swapper/008:55:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional