You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-27 - 14:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Fri Mar 27, 2026 12:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
278172233163,57sleep10-21swapper/107:06:001
280392230161,56sleep20-21swapper/207:08:552
279432202165,25sleep30-21swapper/307:07:383
277842200165,24sleep00-21swapper/007:05:350
797121530,4sleep12821799cyclictest12:30:011
28231998076,3cyclictest0-21swapper/312:03:563
28231998076,3cyclictest0-21swapper/310:38:023
28231998076,3cyclictest0-21swapper/307:39:323
137242760,5sleep32823199cyclictest09:00:193
28231997571,3cyclictest0-21swapper/310:52:013
28231997571,3cyclictest0-21swapper/310:52:013
28231997268,3cyclictest0-21swapper/312:16:463
28231997066,3cyclictest0-21swapper/312:32:393
225682680,2sleep30-21swapper/309:20:173
28231995855,2cyclictest0-21swapper/311:54:303
296102540,4sleep011-21rcuc/012:05:120
28231994945,3cyclictest0-21swapper/310:20:223
199382480,2sleep230-21rcuc/209:15:142
74292450,4sleep07433-21switchtime08:45:230
28217994024,10cyclictest0-21swapper/110:40:141
2823199382,24cyclictest18866-21kworker/3:111:49:513
2823199381,28cyclictest18866-21kworker/3:109:27:023
2821199368,13cyclictest3737-21munin-run08:40:010
2821199358,12cyclictest18167-21awk11:40:000
2821199358,12cyclictest10331-21awk08:55:020
28211993510,12cyclictest32315-21munin-run09:45:010
28211993510,12cyclictest32315-21munin-run09:45:010
28231993422,2cyclictest18866-21kworker/3:109:42:183
28231993422,2cyclictest18866-21kworker/3:109:42:183
2822599341,24cyclictest0-21swapper/208:31:572
2822599341,24cyclictest0-21swapper/208:31:572
2823199331,23cyclictest0-21swapper/311:34:483
2821799330,11cyclictest0-21swapper/112:30:541
2821799330,10cyclictest0-21swapper/111:01:031
28231993216,2cyclictest18866-21kworker/3:107:29:303
2823199321,23cyclictest0-21swapper/310:27:163
2823199321,23cyclictest0-21swapper/309:47:543
2822599321,13cyclictest0-21swapper/208:45:402
2821799320,12cyclictest0-21swapper/112:02:261
2823199311,23cyclictest0-21swapper/312:07:063
2823199311,23cyclictest0-21swapper/311:59:023
2822599301,14cyclictest0-21swapper/208:16:532
2822599300,23cyclictest0-21swapper/209:03:122
2821799304,3cyclictest0-21swapper/110:33:221
28217993020,3cyclictest377-21smart_sda12:10:221
2821799301,20cyclictest0-21swapper/110:25:121
2823199292,10cyclictest18866-21kworker/3:108:48:193
2822599291,11cyclictest0-21swapper/207:21:252
2822599290,10cyclictest0-21swapper/208:04:212
2821799291,6cyclictest0-21swapper/109:59:591
28211992910,16cyclictest20351-21munin-run11:45:000
301912280,2sleep10-21swapper/110:50:251
301912280,2sleep10-21swapper/110:50:251
2823199282,9cyclictest18866-21kworker/3:109:58:593
2823199281,8cyclictest18866-21kworker/3:110:47:193
2822599281,9cyclictest0-21swapper/208:26:252
2822599281,9cyclictest0-21swapper/207:16:002
2822599281,9cyclictest0-21swapper/207:16:002
2822599281,10cyclictest0-21swapper/208:40:052
2822599281,10cyclictest0-21swapper/207:57:572
2822599281,10cyclictest0-21swapper/207:44:372
2822599280,9cyclictest0-21swapper/209:41:492
2822599280,9cyclictest0-21swapper/209:41:492
2822599280,2cyclictest0-21swapper/208:39:572
2822599280,13cyclictest0-21swapper/212:06:052
2822599280,13cyclictest0-21swapper/212:01:172
2822599280,12cyclictest0-21swapper/211:06:282
2822599280,12cyclictest0-21swapper/208:14:012
2822599280,11cyclictest0-21swapper/209:48:412
2821799282,2cyclictest86750irq/128-enp1s0-11:06:391
2821799280,26cyclictest0-21swapper/110:08:071
2823199279,2cyclictest18866-21kworker/3:107:58:263
2822599271,12cyclictest0-21swapper/211:38:372
2822599271,12cyclictest0-21swapper/210:41:042
2822599271,11cyclictest0-21swapper/211:17:012
2822599271,10cyclictest0-21swapper/211:56:012
2822599270,13cyclictest0-21swapper/210:05:492
2822599270,13cyclictest0-21swapper/209:13:442
2822599270,11cyclictest0-21swapper/210:21:532
2822599270,10cyclictest0-21swapper/211:53:332
2821799276,12cyclictest0-21swapper/109:40:151
2821799276,12cyclictest0-21swapper/109:40:151
2821799273,3cyclictest86750irq/128-enp1s0-09:11:391
2821799270,8cyclictest0-21swapper/110:38:021
2823199261,19cyclictest0-21swapper/312:23:503
2823199261,16cyclictest0-21swapper/311:14:283
2823199261,11cyclictest18866-21kworker/3:108:57:473
28231992610,3cyclictest18866-21kworker/3:112:25:273
2823199260,9cyclictest18866-21kworker/3:109:54:223
2822599261,9cyclictest0-21swapper/210:19:572
2822599261,9cyclictest0-21swapper/209:27:362
2822599261,9cyclictest0-21swapper/208:50:442
2822599261,12cyclictest0-21swapper/212:31:012
2822599261,11cyclictest0-21swapper/209:20:282
2822599261,10cyclictest0-21swapper/212:36:162
2822599261,10cyclictest0-21swapper/208:05:032
2822599260,12cyclictest0-21swapper/210:25:052
2822599260,11cyclictest0-21swapper/212:14:452
2822599260,10cyclictest0-21swapper/211:23:252
2822599260,10cyclictest0-21swapper/211:01:162
2822599260,10cyclictest0-21swapper/210:50:282
2822599260,10cyclictest0-21swapper/210:50:282
2822599260,10cyclictest0-21swapper/210:48:002
2821799262,2cyclictest86750irq/128-enp1s0-09:46:591
2821799261,4cyclictest32145-21cut07:15:181
2821799261,4cyclictest32145-21cut07:15:181
2821799260,6cyclictest0-21swapper/111:49:511
37122250,4sleep2311ktimersoftd/207:25:152
2823199252,16cyclictest18866-21kworker/3:107:47:183
2823199251,21cyclictest0-21swapper/310:55:273
2823199251,16cyclictest0-21swapper/310:40:413
2823199251,16cyclictest0-21swapper/310:19:433
28231992510,4cyclictest18866-21kworker/3:107:50:143
2822599251,9cyclictest0-21swapper/212:27:412
2822599251,9cyclictest0-21swapper/208:56:362
2822599251,11cyclictest0-21swapper/212:20:092
2822599251,11cyclictest0-21swapper/210:32:012
2822599250,9cyclictest0-21swapper/211:42:092
2822599250,9cyclictest0-21swapper/211:12:252
2822599250,11cyclictest0-21swapper/212:18:332
2822599250,11cyclictest0-21swapper/211:29:052
2822599250,11cyclictest0-21swapper/210:58:322
2822599250,11cyclictest0-21swapper/207:45:572
2822599250,10cyclictest0-21swapper/208:21:012
28217992519,3cyclictest15302-21dnf09:05:141
28217992518,4cyclictest1752-21df12:15:131
2821799251,7cyclictest0-21swapper/109:15:471
2821799251,17cyclictest0-21swapper/109:32:291
2821799251,17cyclictest0-21swapper/109:00:251
2821799251,14cyclictest0-21swapper/111:14:311
2821799250,9cyclictest0-21swapper/107:30:191
2821799250,7cyclictest0-21swapper/112:36:581
2821799250,11cyclictest0-21swapper/111:55:551
2821799250,11cyclictest0-21swapper/110:58:191
2821199251,13cyclictest26345-21awk10:45:010
252942250,1sleep00-21swapper/008:15:000
2823199243,3cyclictest86650irq/127-enp1s0-08:53:473
28231992418,3cyclictest6769-21/usr/sbin/munin10:00:183
2823199241,15cyclictest86550irq/126-enp1s0-10:10:193
2823199241,15cyclictest0-21swapper/310:31:223
2823199241,11cyclictest19077-21sshd09:14:353
28225992417,4cyclictest6206-21sed07:30:162
2822599241,10cyclictest0-21swapper/209:06:522
2822599240,9cyclictest0-21swapper/209:55:372
2822599240,12cyclictest0-21swapper/211:48:092
2822599240,12cyclictest0-21swapper/210:10:052
2822599240,11cyclictest0-21swapper/209:37:252
2822599240,11cyclictest0-21swapper/209:37:252
2822599240,11cyclictest0-21swapper/209:30:562
2821799249,8cyclictest0-21swapper/110:10:271
2821799248,9cyclictest0-21swapper/108:45:181
2821799248,3cyclictest0-21swapper/108:07:151
2821799244,11cyclictest2743-21cut08:35:181
2821799242,3cyclictest31700-21munin-run08:30:001
2821799241,8cyclictest0-21swapper/110:21:021
2821799241,8cyclictest0-21swapper/109:27:311
28217992418,3cyclictest16538-21bc07:55:121
28217992413,4cyclictest0-21swapper/110:48:121
2821799241,16cyclictest0-21swapper/111:39:441
2821799241,11cyclictest645-21ps08:30:221
2821799241,11cyclictest645-21ps08:30:221
2821799241,11cyclictest0-21swapper/111:50:151
2821199249,12cyclictest4973-21munin-run11:10:010
2821199242,11cyclictest18508-21awk08:00:020
28211992410,3cyclictest86750irq/128-enp1s0-11:50:150
2823199239,2cyclictest18866-21kworker/3:107:20:223
2823199236,2cyclictest18866-21kworker/3:108:40:473
2823199235,6cyclictest771-21rngd11:25:113
2823199232,6cyclictest7512-21munin-run07:35:003
2823199231,6cyclictest0-21swapper/312:11:093
2823199231,15cyclictest0-21swapper/309:19:013
2823199231,12cyclictest0-21swapper/311:09:503
2823199231,11cyclictest28377-21cstates09:35:113
2823199231,11cyclictest28377-21cstates09:35:113
2823199231,10cyclictest0-21swapper/311:22:543
2822599235,3cyclictest0-21swapper/207:14:532
2822599235,3cyclictest0-21swapper/207:14:532
2822599231,9cyclictest0-21swapper/211:33:092
2822599231,11cyclictest0-21swapper/207:39:532
2822599230,9cyclictest0-21swapper/209:51:572
2821799237,3cyclictest0-21swapper/111:40:351
2821799233,3cyclictest0-21swapper/109:23:551
2821799233,15cyclictest6351-21kworker/u8:210:15:471
2821799232,9cyclictest0-21swapper/111:18:301
28217992319,3cyclictest0-21swapper/108:55:271
28217992319,3cyclictest0-21swapper/108:41:081
28217992319,3cyclictest0-21swapper/108:00:091
2821799231,3cyclictest86750irq/128-enp1s0-07:26:151
2821799231,10cyclictest30572-21open_files07:15:021
2821799231,10cyclictest30572-21open_files07:15:011
2821799230,11cyclictest0-21swapper/109:54:351
2821199235,15cyclictest480-21awk11:00:010
2821199231,11cyclictest28547-21awk10:50:010
2823199228,3cyclictest31305-21/usr/sbin/munin07:15:133
2823199228,3cyclictest31305-21/usr/sbin/munin07:15:133
2823199228,3cyclictest28417-21/usr/sbin/munin07:10:133
2823199228,3cyclictest28417-21/usr/sbin/munin07:10:133
2823199224,3cyclictest86850irq/129-enp1s0-09:08:193
2823199223,4cyclictest0-21swapper/311:35:283
2823199222,9cyclictest2380-21users11:00:253
2823199221,14cyclictest0-21swapper/309:30:363
2823199221,11cyclictest0-21swapper/311:17:463
2823199221,10cyclictest31489-21sshd08:27:343
2823199220,20cyclictest0-21swapper/311:40:243
2822599221,9cyclictest0-21swapper/207:52:292
2822599220,9cyclictest0-21swapper/210:37:252
2822599220,8cyclictest0-21swapper/210:03:572
2821799228,9cyclictest0-21swapper/111:27:081
2821799228,3cyclictest0-21swapper/110:01:471
2821799224,3cyclictest0-21swapper/111:23:151
2821799224,3cyclictest0-21swapper/107:54:191
2821799223,3cyclictest86850irq/129-enp1s0-11:30:111
28217992214,2cyclictest0-21swapper/107:40:131
2821799221,3cyclictest86750irq/128-enp1s0-08:51:231
2821799221,2cyclictest812-21snmpd07:36:541
2821799221,18cyclictest29613-21cat12:05:121
2821799221,10cyclictest29205-21ntpq09:35:211
2821799221,10cyclictest29205-21ntpq09:35:211
2823199218,2cyclictest18866-21kworker/3:108:00:023
2823199216,2cyclictest18866-21kworker/3:108:11:033
2823199211,13cyclictest0-21swapper/312:38:113
2823199211,11cyclictest18866-21kworker/3:108:21:473
2823199210,6cyclictest9901-21wc10:05:183
2821799212,2cyclictest86750irq/128-enp1s0-08:23:211
2821799211,18cyclictest0-21swapper/108:10:161
2821799211,15cyclictest0-21swapper/107:22:211
2821799210,14cyclictest0-21swapper/112:20:471
2821199211,11cyclictest4297-21sshd09:54:330
2821199211,11cyclictest19351-21/usr/sbin/munin09:15:120
2821199210,18cyclictest13773-21idleruntime-cro11:30:020
2823199207,10cyclictest3447-21sshd08:36:343
2821799208,3cyclictest0-21swapper/107:46:171
2821799201,18cyclictest0-21swapper/108:15:091
2821199201,12cyclictest0-21swapper/010:11:410
2821199201,11cyclictest9934-21/usr/sbin/munin07:40:260
2823199195,3cyclictest18866-21kworker/3:108:06:143
2823199193,3cyclictest18866-21kworker/3:107:42:183
2823199191,9cyclictest0-21swapper/308:32:473
2823199191,9cyclictest0-21swapper/308:32:473
2821199193,3cyclictest0-21swapper/008:30:150
2821199193,3cyclictest0-21swapper/008:30:140
2821199191,6cyclictest0-21swapper/009:23:580
28211991913,4cyclictest7727-21df11:15:130
28211991913,3cyclictest4842-21grep07:25:260
2821199191,15cyclictest6769-21/usr/sbin/munin10:00:180
2821199191,15cyclictest32498-21memory12:10:180
2821199191,10cyclictest1396-21/usr/sbin/munin12:15:240
2821199191,10cyclictest10207-21/usr/sbin/munin12:35:140
2823199181,5cyclictest26584-21smart_sda08:15:233
2821199187,3cyclictest812-21snmpd10:32:300
2821199184,4cyclictest30733-21munin-run10:55:010
2821199184,4cyclictest30733-21munin-run10:55:010
2821199182,7cyclictest20920-21df_inode11:45:130
2821199181,9cyclictest25322-21/usr/sbin/munin08:15:230
2821199181,8cyclictest0-21swapper/009:31:390
2821199181,7cyclictest31305-21/usr/sbin/munin07:15:250
2821199181,7cyclictest31305-21/usr/sbin/munin07:15:250
2821199181,6cyclictest9466-21timerwakeupswit12:30:260
28211991813,4cyclictest0-21swapper/012:03:470
28211991811,4cyclictest15946-21cut09:05:190
2821199181,13cyclictest8978-21/usr/sbin/munin10:05:120
2821199181,11cyclictest0-21swapper/007:13:340
2821199181,11cyclictest0-21swapper/007:13:340
2821199180,5cyclictest86550irq/126-enp1s0-10:28:180
57172171,2sleep00-21swapper/008:42:310
2821199177,7cyclictest5161-21unixbench_singl12:20:250
2821199173,4cyclictest22540-21sshd08:05:330
2821199173,3cyclictest15201-21/usr/sbin/munin07:50:190
2821199172,4cyclictest13984-21/usr/sbin/munin11:30:180
2821199172,3cyclictest0-21swapper/011:12:380
2821199172,12cyclictest28518-21head08:20:180
2821199171,7cyclictest0-21swapper/010:24:050
2821199171,6cyclictest0-21swapper/010:15:250
2821199171,5cyclictest0-21swapper/009:25:310
2821199171,3cyclictest24972-21/usr/sbin/munin11:55:140
28211991712,3cyclictest17769-21gltestperf09:10:150
2821199171,15cyclictest0-21swapper/009:38:520
2821199171,15cyclictest0-21swapper/009:38:520
2821199171,15cyclictest0-21swapper/007:37:090
2821199171,14cyclictest0-21swapper/008:00:040
28211991711,3cyclictest10445-21grep11:20:180
140602170,3sleep00-21swapper/007:49:420
2821199164,4cyclictest1963-21grep11:00:220
2821199162,11cyclictest4571-21/usr/sbin/munin09:55:160
2821199161,6cyclictest957-21interrupts09:45:170
2821199161,4cyclictest5539-21/usr/sbin/munin07:30:230
2821199161,3cyclictest0-21swapper/010:35:110
2821199161,14cyclictest0-21swapper/009:03:310
28211991611,3cyclictest1521-21cut07:20:140
28211991611,3cyclictest11798-21sendmail_mailtr08:55:240
2821199161,12cyclictest5813-21/usr/sbin/munin12:25:160
28211991510,3cyclictest29722-21/usr/sbin/munin08:25:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional