You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-20 - 18:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sun Sep 20, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30152235161,61sleep00-21swapper/007:06:530
31662204163,28sleep20-21swapper/207:08:522
29882203167,24sleep30-21swapper/307:06:313
29242203167,24sleep10-21swapper/107:05:411
3234421510,4sleep2335599cyclictest09:25:242
3267621390,5sleep0334599cyclictest10:40:180
3355991080,107cyclictest0-21swapper/212:14:332
3355991070,105cyclictest0-21swapper/211:31:332
3355998986,2cyclictest0-21swapper/207:26:062
3355997370,2cyclictest0-21swapper/209:14:172
326712510,1sleep10-21swapper/109:25:271
39652490,1sleep10-21swapper/109:35:181
41222480,2sleep00-21swapper/008:20:250
77142470,1sleep10-21swapper/109:45:141
185532470,4sleep341-21ksoftirqd/312:35:183
334599401,36cyclictest24429-21idleruntime-cro09:10:010
334599397,22cyclictest23683-21/usr/sbin/munin11:35:250
334599391,26cyclictest12463-21awk12:20:240
334599391,25cyclictest8497-21kernelversion09:45:200
3345993910,13cyclictest17829-21basename08:55:000
3355993723,8cyclictest0-21swapper/211:54:302
334599371,26cyclictest28284-21kernelversion10:30:190
334599371,26cyclictest28284-21kernelversion10:30:180
334599365,22cyclictest6730-21/usr/sbin/munin12:10:200
334599362,24cyclictest13647-21/usr/sbin/munin08:45:110
71192350,4sleep1334999cyclictest07:15:171
334599356,13cyclictest27869-21basename11:45:010
334599356,13cyclictest27869-21basename11:45:010
334599352,23cyclictest784-21latency_hist09:30:020
334599346,18cyclictest87550irq/128-enp1s0-07:15:230
334599345,14cyclictest12050irq/123-ahci[0010:50:210
334599341,7cyclictest9869-21/usr/sbin/munin09:50:130
334599341,30cyclictest16879-21awk11:20:010
334599341,23cyclictest12701-21/usr/sbin/munin11:10:130
335599330,23cyclictest0-21swapper/207:11:412
334599339,16cyclictest14270-21/usr/sbin/munin10:00:140
334599333,19cyclictest87350irq/126-enp1s0-10:25:140
334599332,27cyclictest25085-21ntp_states07:55:230
334599331,29cyclictest17709-21/usr/sbin/munin12:35:250
334599331,29cyclictest11452-21/usr/sbin/munin08:40:150
334599331,21cyclictest26612-21latency_hist09:15:010
334599325,14cyclictest87550irq/128-enp1s0-08:35:260
334599322,17cyclictest20496-21taskset10:13:180
334599321,2cyclictest87350irq/126-enp1s0-11:25:240
334599321,27cyclictest13899-40dnf09:58:210
334599321,23cyclictest4530-21/usr/sbin/munin12:05:260
334599321,22cyclictest808-21snmpd09:38:360
334599317,21cyclictest9037-21cpuspeed_turbos12:15:120
334599311,22cyclictest808-21snmpd08:09:200
334599311,22cyclictest808-21snmpd08:09:200
334599310,28cyclictest6097-21/usr/sbin/munin10:55:220
334599305,12cyclictest22225-21munin-run09:05:010
334599303,15cyclictest87550irq/128-enp1s0-07:45:140
334599303,14cyclictest17092-21/usr/sbin/munin11:20:170
334599302,25cyclictest8643-21diskstats11:00:150
334599302,25cyclictest30279-21/usr/sbin/munin11:50:160
334599301,3cyclictest29323-21sendmail_mailst11:45:240
334599301,3cyclictest15864-21diskstats12:30:160
334599301,26cyclictest29819-21/usr/sbin/munin09:20:170
334599301,26cyclictest29802-21cut10:35:130
334599301,25cyclictest11740-21sendmail_mailqu11:05:240
335599297,9cyclictest1284-21wc08:15:192
335599291,11cyclictest0-21swapper/210:18:052
335599290,13cyclictest0-21swapper/212:21:052
334999291,13cyclictest0-21swapper/111:30:031
334999291,11cyclictest0-21swapper/110:10:031
334599292,3cyclictest2314-21/usr/sbin/munin12:00:190
334599292,24cyclictest27915-21ntp_states09:15:220
334599292,24cyclictest27915-21ntp_states09:15:220
334599291,4cyclictest1717-21/usr/sbin/munin11:55:270
334599291,3cyclictest17219-21/usr/sbin/munin10:05:170
334599291,3cyclictest13640-21expr12:25:140
334599291,25cyclictest5815-21/usr/sbin/munin09:40:140
334599291,25cyclictest26562-21diskstats08:00:150
334599291,25cyclictest1055-21/usr/sbin/munin09:30:260
334599291,17cyclictest24415-21diskmemload11:30:320
334599290,4cyclictest0-21swapper/007:30:130
335599281,9cyclictest0-21swapper/208:13:252
335599281,11cyclictest808-21snmpd12:09:052
335599281,11cyclictest12414-21df_inode09:55:132
335599280,16cyclictest0-21swapper/211:44:442
335599280,16cyclictest0-21swapper/211:44:442
335599280,12cyclictest0-21swapper/211:09:572
335599280,12cyclictest0-21swapper/209:37:262
335599280,10cyclictest0-21swapper/210:43:412
335599280,10cyclictest0-21swapper/210:24:252
334599289,16cyclictest11812-21needreboot07:25:200
334599286,19cyclictest4693-21grep07:10:230
334599285,20cyclictest22133-21smart_sda10:15:250
334599282,23cyclictest24103-21/usr/sbin/munin10:20:200
334599281,4cyclictest18043-21/usr/sbin/munin08:55:160
334599281,24cyclictest7612-21perl08:30:180
334599281,24cyclictest10604-21latency_hist07:25:020
3345992812,13cyclictest16760-21grep07:35:270
334599281,17cyclictest0-21swapper/010:45:230
3345992810,15cyclictest17545-21grep07:40:120
334599280,25cyclictest346-21/usr/sbin/munin08:15:140
335599271,9cyclictest0-21swapper/209:43:362
335599271,9cyclictest0-21swapper/207:50:332
335599271,18cyclictest0-21swapper/210:00:172
335599271,12cyclictest0-21swapper/212:26:532
335599271,10cyclictest0-21swapper/212:34:332
335599271,10cyclictest0-21swapper/207:41:552
335599271,10cyclictest0-21swapper/207:22:092
335599270,24cyclictest0-21swapper/211:26:212
335599270,17cyclictest0-21swapper/211:39:162
335599270,13cyclictest0-21swapper/209:49:402
335599270,11cyclictest0-21swapper/211:17:252
335599270,11cyclictest0-21swapper/209:52:372
335599270,10cyclictest0-21swapper/208:29:332
334999271,18cyclictest11181-21hddtemp_smartct11:05:161
334999271,13cyclictest0-21swapper/111:12:301
334999271,10cyclictest0-21swapper/109:05:351
334599272,22cyclictest22492-21hddtemp_smartct07:50:160
334599271,23cyclictest909-21sssd_nss08:25:150
335599261,11cyclictest0-21swapper/208:02:172
335599260,11cyclictest0-21swapper/210:46:372
335599260,11cyclictest0-21swapper/208:34:412
335599260,10cyclictest0-21swapper/210:10:292
334999261,9cyclictest0-21swapper/108:55:231
334999261,10cyclictest0-21swapper/107:28:191
334599261,21cyclictest30714-21grep08:10:140
335599254,14cyclictest0-21swapper/209:09:312
335599251,9cyclictest11788-21df08:40:152
335599251,9cyclictest0-21swapper/207:55:552
335599251,7cyclictest0-21swapper/208:22:472
335599251,10cyclictest0-21swapper/211:01:492
335599251,10cyclictest0-21swapper/209:32:292
335599251,10cyclictest0-21swapper/209:22:012
335599250,9cyclictest0-21swapper/209:02:092
335599250,8cyclictest0-21swapper/211:49:012
335599250,2cyclictest0-21swapper/208:07:492
335599250,2cyclictest0-21swapper/208:07:492
335599250,17cyclictest0-21swapper/209:15:372
335599250,17cyclictest0-21swapper/209:15:372
335599250,12cyclictest0-21swapper/210:30:332
335599250,12cyclictest0-21swapper/210:30:332
335599250,10cyclictest0-21swapper/210:58:172
335599250,10cyclictest0-21swapper/207:38:012
334999251,9cyclictest0-21swapper/107:40:071
3349992512,11cyclictest16508-21kworker/u8:011:35:081
334999251,11cyclictest0-21swapper/111:17:311
334999251,10cyclictest0-21swapper/111:29:031
334999251,10cyclictest0-21swapper/108:37:111
334999251,10cyclictest0-21swapper/107:52:431
334999250,10cyclictest0-21swapper/112:12:071
334999250,10cyclictest0-21swapper/109:14:471
119052250,2sleep30-21swapper/307:25:223
336199241,5cyclictest0-21swapper/311:57:173
335599241,9cyclictest0-21swapper/211:14:292
335599241,9cyclictest0-21swapper/210:07:512
335599241,9cyclictest0-21swapper/207:18:152
335599241,8cyclictest0-21swapper/208:57:012
335599240,11cyclictest0-21swapper/211:20:292
335599240,11cyclictest0-21swapper/210:52:132
335599240,11cyclictest0-21swapper/210:38:212
3349992411,6cyclictest7474-21kworker/u8:207:55:231
334999241,10cyclictest0-21swapper/109:43:271
334999240,11cyclictest0-21swapper/111:46:151
336199231,6cyclictest0-21swapper/311:10:213
335599231,9cyclictest0-21swapper/212:16:072
335599231,9cyclictest0-21swapper/210:28:532
335599230,9cyclictest0-21swapper/211:56:332
335599230,9cyclictest0-21swapper/208:45:552
335599230,10cyclictest0-21swapper/207:33:162
334999231,9cyclictest0-21swapper/111:50:551
334999231,9cyclictest0-21swapper/109:17:541
334999231,9cyclictest0-21swapper/109:17:541
3349992313,3cyclictest0-21swapper/110:07:531
334999231,19cyclictest17196-21munin-run07:40:021
334999231,11cyclictest0-21swapper/111:58:391
334999231,11cyclictest0-21swapper/110:48:511
334999231,11cyclictest0-21swapper/107:30:071
334999231,10cyclictest0-21swapper/111:02:341
334999231,10cyclictest0-21swapper/110:25:551
334999231,10cyclictest0-21swapper/109:55:351
334999231,10cyclictest0-21swapper/109:54:271
334999230,9cyclictest0-21swapper/110:15:031
334999230,10cyclictest0-21swapper/111:44:391
334999230,10cyclictest0-21swapper/111:44:391
336199221,11cyclictest0-21swapper/311:50:023
335599225,3cyclictest0-21swapper/207:45:112
335599221,10cyclictest0-21swapper/212:36:412
335599221,10cyclictest0-21swapper/212:01:192
335599221,10cyclictest0-21swapper/208:35:332
3349992214,5cyclictest17762-21systemd-journal08:54:291
334999221,12cyclictest10186-21smart_sda12:15:241
334999221,10cyclictest0-21swapper/109:30:591
334999221,10cyclictest0-21swapper/108:42:271
334999221,10cyclictest0-21swapper/108:31:431
334999221,10cyclictest0-21swapper/108:05:071
334999221,10cyclictest0-21swapper/108:05:071
3349992210,4cyclictest15080-21irqstats10:00:181
3349992210,3cyclictest4303-21munin-run12:05:011
336199219,3cyclictest28081-21/usr/sbin/munin11:45:193
336199212,10cyclictest0-21swapper/309:25:143
336199211,9cyclictest10841-21users08:35:273
336199211,6cyclictest0-21swapper/310:12:413
336199211,4cyclictest8443-21kworker/u8:112:14:073
336199211,11cyclictest18154-21ntp_states11:20:203
334999211,9cyclictest0-21swapper/108:13:431
334999211,3cyclictest827-21runrttasks10:40:121
334999211,15cyclictest0-21swapper/110:22:061
334999211,10cyclictest0-21swapper/112:39:351
334999211,10cyclictest0-21swapper/110:56:231
334999211,10cyclictest0-21swapper/110:30:431
334999211,10cyclictest0-21swapper/110:30:431
334999210,9cyclictest0-21swapper/112:33:271
334999210,9cyclictest0-21swapper/111:20:511
334999210,9cyclictest0-21swapper/110:51:471
336199206,6cyclictest0-21swapper/312:15:193
336199201,6cyclictest26237-21users09:10:263
336199201,6cyclictest0-21swapper/310:02:173
3361992014,4cyclictest27512-21cut09:15:163
3361992014,4cyclictest27512-21cut09:15:163
336199201,10cyclictest24415-21diskmemload10:20:243
336199201,10cyclictest13031-21/usr/sbin/munin07:30:153
336199201,10cyclictest0-21swapper/309:21:423
336199200,13cyclictest0-21swapper/309:33:193
335599207,3cyclictest0-21swapper/208:50:232
334999209,3cyclictest27476-21smart_sda08:00:251
334999202,15cyclictest14528-21grep08:45:181
334999201,9cyclictest0-21swapper/110:36:311
334999201,8cyclictest0-21swapper/112:08:071
334999201,10cyclictest5771-21memory08:25:201
334999200,2cyclictest87650irq/129-enp1s0-09:02:201
334999200,10cyclictest0-21swapper/112:23:031
334999200,10cyclictest0-21swapper/109:22:591
168352201,3sleep317058-21fw_packets10:05:143
336199198,3cyclictest808-21snmpd11:39:003
336199197,5cyclictest39-21rcuc/307:40:023
336199195,8cyclictest19791-40dnf08:57:233
336199194,7cyclictest0-21swapper/309:55:133
336199192,6cyclictest5563-21munin-node07:15:013
336199192,5cyclictest0-21swapper/312:29:123
336199192,15cyclictest0-21swapper/310:45:133
336199191,7cyclictest0-21swapper/312:09:563
336199191,7cyclictest0-21swapper/309:38:083
336199191,6cyclictest8564-1kworker/u9:110:31:333
336199191,6cyclictest8564-1kworker/u9:110:31:333
336199191,5cyclictest25301-21switchtime07:55:243
336199191,4cyclictest0-21swapper/310:15:283
336199191,2cyclictest29684-21smart_sda08:05:233
336199191,2cyclictest29684-21smart_sda08:05:233
336199191,15cyclictest32297-21fschecks_count10:40:153
336199191,11cyclictest14248-21gltestperf08:45:163
336199191,10cyclictest4743-21kernelversion10:50:193
336199191,10cyclictest2648-21/usr/sbin/munin08:20:133
334999199,9cyclictest0-21swapper/112:25:241
3349991915,3cyclictest0-21swapper/107:48:151
334999191,16cyclictest0-21swapper/108:21:541
334999190,17cyclictest0-21swapper/108:19:231
336199185,4cyclictest21133-21sed09:00:203
336199181,8cyclictest0-21swapper/312:33:453
336199181,7cyclictest9368-21ntp_states11:00:223
336199181,7cyclictest6514-21tr09:40:213
336199181,7cyclictest0-21swapper/311:33:583
336199181,6cyclictest32589-21munin-run08:15:003
336199181,6cyclictest0-21swapper/309:10:013
336199181,3cyclictest0-21swapper/309:48:243
3361991813,3cyclictest13007-21head08:40:283
3361991812,4cyclictest10975-21cut11:05:163
336199181,10cyclictest24415-21diskmemload10:35:183
334999181,16cyclictest0-21swapper/107:21:241
334999181,16cyclictest0-21swapper/107:11:501
336199176,3cyclictest87350irq/126-enp1s0-11:40:273
336199176,3cyclictest87350irq/126-enp1s0-11:40:273
336199176,3cyclictest87350irq/126-enp1s0-09:50:263
336199173,4cyclictest25818-21grep10:25:163
336199172,5cyclictest8564-1kworker/u9:112:22:073
336199172,4cyclictest18484-21ntp_states07:40:193
336199171,7cyclictest909-21sssd_nss11:15:163
336199171,14cyclictest0-21swapper/311:25:273
3361991711,3cyclictest7647-21grep10:55:263
336199171,13cyclictest2784-21grep12:00:153
336199161,6cyclictest8635-21/usr/sbin/munin07:20:193
336199161,6cyclictest6103-21forks07:15:043
336199161,6cyclictest0-21swapper/308:52:483
336199161,2cyclictest362-21cpu08:15:133
3361991612,3cyclictest0-21swapper/308:32:463
336199161,12cyclictest26214-21/usr/sbin/munin08:00:213
3361991610,3cyclictest22041-21ls07:50:123
193812161,2sleep30-21swapper/307:45:003
336199155,3cyclictest87350irq/126-enp1s0-08:25:263
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional