You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-11 - 15:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3s.osadl.org (updated Thu Jun 11, 2026 12:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
69312234161,60sleep00-21swapper/007:06:370
70272233158,24sleep30-21swapper/307:07:543
69952232164,55sleep20-21swapper/207:07:282
69122203166,24sleep10-21swapper/107:06:221
218821540,5sleep3729899cyclictest09:20:253
7292991060,2cyclictest0-21swapper/207:59:212
7292999693,2cyclictest0-21swapper/211:09:162
7302610,1sleep20-21swapper/209:20:122
194822460,1sleep10-21swapper/107:35:121
7292993431,2cyclictest0-21swapper/212:12:182
7288993327,3cyclictest32687-21df08:05:121
282862330,4sleep3729899cyclictest07:55:133
728899301,8cyclictest0-21swapper/111:34:171
729899291,13cyclictest0-21swapper/310:00:203
728899295,11cyclictest0-21swapper/109:21:291
728899294,22cyclictest32601-21gltestperf11:45:141
728899291,17cyclictest0-21swapper/109:50:101
728199290,3cyclictest8519-21switchtime09:35:240
729899281,12cyclictest0-21swapper/312:24:213
7298992810,11cyclictest0-21swapper/311:15:593
728899282,18cyclictest9766-21kworker/u8:211:02:041
728899282,18cyclictest13789-21kworker/u8:110:07:311
728899281,11cyclictest0-21swapper/110:27:021
729899276,3cyclictest0-21swapper/310:05:143
729899270,15cyclictest0-21swapper/308:09:203
729899270,13cyclictest0-21swapper/309:50:223
728899271,4cyclictest24288-21ntp_states11:25:211
728899271,12cyclictest1-21systemd08:30:021
728899271,12cyclictest0-21swapper/112:33:301
728899271,12cyclictest0-21swapper/112:33:301
728899271,11cyclictest0-21swapper/112:23:471
728199277,17cyclictest29925-21munin-run08:00:010
728199271,14cyclictest8567-21munin-run08:25:020
729899261,13cyclictest0-21swapper/308:45:133
729899261,12cyclictest9318-21chrt07:13:273
729899261,12cyclictest0-21swapper/312:19:343
729899261,12cyclictest0-21swapper/307:48:483
729899261,10cyclictest23105-21cut11:25:013
7298992610,3cyclictest11322-21df08:30:133
728899263,9cyclictest28348-21diskmemload11:39:081
728899263,9cyclictest0-21swapper/110:22:041
728899263,12cyclictest9801-21latency_hist10:55:021
728899262,11cyclictest0-21swapper/109:55:331
728899262,11cyclictest0-21swapper/109:05:361
728899261,4cyclictest0-21swapper/110:19:351
728899261,4cyclictest0-21swapper/110:19:351
728899261,22cyclictest909-21sssd_nss10:10:311
728899261,16cyclictest0-21swapper/111:23:031
728899261,11cyclictest0-21swapper/110:04:141
728899261,11cyclictest0-21swapper/109:46:141
728899261,10cyclictest0-21swapper/110:30:511
728199261,13cyclictest27389-21munin-run11:35:000
729899259,3cyclictest0-21swapper/308:41:273
729899258,3cyclictest0-21swapper/311:36:243
729899258,3cyclictest0-21swapper/307:40:283
729899251,12cyclictest0-21swapper/308:20:593
729899251,11cyclictest0-21swapper/309:08:203
729899251,11cyclictest0-21swapper/308:29:073
729899251,11cyclictest0-21swapper/307:39:203
729899251,10cyclictest0-21swapper/312:25:593
729899251,10cyclictest0-21swapper/309:17:563
729899250,2cyclictest26835-21latency_hist10:20:023
729899250,2cyclictest26835-21latency_hist10:20:023
728899257,3cyclictest0-21swapper/111:05:141
728899257,3cyclictest0-21swapper/107:26:271
728899256,3cyclictest28298-21diskstats07:55:131
728899252,9cyclictest9373-21memory12:05:181
728899252,11cyclictest0-21swapper/108:35:211
728899251,9cyclictest0-21swapper/108:14:541
728899251,9cyclictest0-21swapper/107:46:381
728899251,6cyclictest0-21swapper/109:44:121
728899251,5cyclictest17593-21diskstats12:25:141
728899251,21cyclictest31244-21/usr/sbin/munin09:15:151
728899251,21cyclictest14815-21latency_hist12:20:011
728899251,21cyclictest12961-21ls08:35:021
728899251,12cyclictest0-21swapper/107:40:141
728899251,11cyclictest0-21swapper/112:11:151
728899251,11cyclictest0-21swapper/111:17:011
728899251,11cyclictest0-21swapper/110:40:141
728899251,11cyclictest0-21swapper/110:36:261
728899250,6cyclictest0-21swapper/111:41:421
729899249,3cyclictest7156-21memory12:00:203
729899249,3cyclictest0-21swapper/307:32:363
729899248,3cyclictest0-21swapper/311:30:243
729899248,3cyclictest0-21swapper/310:43:113
729899247,3cyclictest0-21swapper/307:54:083
729899241,10cyclictest0-21swapper/308:51:153
729899241,10cyclictest0-21swapper/308:36:553
729899240,12cyclictest0-21swapper/311:28:203
729899240,12cyclictest0-21swapper/309:13:563
729899240,11cyclictest0-21swapper/310:14:363
729299241,5cyclictest0-21swapper/210:52:142
729299241,4cyclictest0-21swapper/212:25:202
728899249,3cyclictest7125-21sed09:35:011
728899247,3cyclictest0-21swapper/107:20:241
728899241,9cyclictest0-21swapper/108:51:161
728899241,7cyclictest0-21swapper/112:00:321
728899241,5cyclictest0-21swapper/111:57:391
728899241,5cyclictest0-21swapper/109:37:501
728899241,4cyclictest0-21swapper/109:26:131
7288992412,3cyclictest0-21swapper/107:10:091
728899241,21cyclictest0-21swapper/108:00:391
728899241,20cyclictest0-21swapper/108:40:171
728899241,12cyclictest0-21swapper/111:50:101
728899241,11cyclictest17071-21grep07:30:111
728899241,11cyclictest0-21swapper/109:00:271
728899241,10cyclictest7837-21sendmail_mailst08:20:241
728899241,10cyclictest0-21swapper/108:58:091
728899240,9cyclictest0-21swapper/108:17:091
728899240,11cyclictest0-21swapper/108:47:311
728199246,15cyclictest15164-21awk08:40:010
729899239,3cyclictest0-21swapper/308:11:403
729899239,3cyclictest0-21swapper/307:25:043
729899238,3cyclictest0-21swapper/311:44:203
729899238,3cyclictest0-21swapper/310:53:353
729899238,3cyclictest0-21swapper/310:30:473
729899238,3cyclictest0-21swapper/309:38:523
729899238,3cyclictest0-21swapper/307:19:433
7298992310,3cyclictest0-21swapper/312:12:403
7298992310,2cyclictest0-21swapper/312:32:573
7298992310,2cyclictest0-21swapper/312:32:573
729299239,3cyclictest31797-21awk11:45:022
729299232,11cyclictest5304-21smart_sda11:55:232
729299231,9cyclictest0-21swapper/212:15:492
729299231,5cyclictest0-21swapper/209:59:042
729299231,11cyclictest14406-21runrttasks07:23:252
729299231,11cyclictest0-21swapper/212:08:222
729299231,11cyclictest0-21swapper/210:08:002
728899235,4cyclictest0-21swapper/110:47:271
728899231,20cyclictest17893-21smart_sda11:10:241
728899231,10cyclictest10050-21grep10:55:121
728899231,10cyclictest0-21swapper/112:35:101
728899231,10cyclictest0-21swapper/109:10:591
7281992316,4cyclictest13068-21grep09:45:230
728199231,5cyclictest3126-21cat08:10:180
728199231,11cyclictest31799-21perf11:45:010
729899228,3cyclictest0-21swapper/311:46:243
729899228,3cyclictest0-21swapper/308:57:233
729899228,3cyclictest0-21swapper/307:21:283
729899227,3cyclictest0-21swapper/309:57:483
729899220,12cyclictest0-21swapper/309:47:163
729299223,9cyclictest7536-21sshd08:20:192
7292992219,2cyclictest0-21swapper/209:14:192
729299221,7cyclictest10348-21sed08:25:242
729299221,7cyclictest0-21swapper/210:55:132
729299221,6cyclictest0-21swapper/211:24:192
729299221,6cyclictest0-21swapper/211:11:492
729299221,6cyclictest0-21swapper/210:39:522
729299221,5cyclictest0-21swapper/209:42:542
729299221,3cyclictest0-21swapper/210:45:242
729299221,12cyclictest0-21swapper/210:25:342
729299221,11cyclictest808-21snmpd08:47:552
728899222,8cyclictest0-21swapper/107:16:461
728899221,19cyclictest0-21swapper/107:51:001
728199227,9cyclictest755-21rngd12:30:220
728199227,9cyclictest755-21rngd12:30:220
728199220,6cyclictest0-21swapper/012:10:380
729899217,3cyclictest0-21swapper/311:59:483
729899217,3cyclictest0-21swapper/311:14:393
729899217,3cyclictest0-21swapper/310:37:473
729899217,3cyclictest0-21swapper/308:19:263
729899216,8cyclictest0-21swapper/308:03:343
729899215,3cyclictest0-21swapper/309:31:523
729299215,8cyclictest12050irq/123-ahci[0007:45:002
729299214,8cyclictest87350irq/126-enp1s0-12:23:112
729299214,8cyclictest7274-21cyclictest10:10:442
729299212,6cyclictest8019-21irqstats09:35:182
729299211,7cyclictest30906-21if_enp2s008:00:162
729299211,7cyclictest19895-21ntp_states11:15:212
729299211,6cyclictest980-21latency_hist10:35:012
729299211,6cyclictest26160-21perf09:05:012
729299211,6cyclictest0-21swapper/210:15:122
729299211,6cyclictest0-21swapper/210:15:122
729299211,6cyclictest0-21swapper/209:49:322
729299211,6cyclictest0-21swapper/209:34:202
729299211,13cyclictest0-21swapper/207:34:382
729299211,10cyclictest8841-21cat07:10:242
729299211,10cyclictest383-21grep11:45:172
729299211,10cyclictest31495-21basename09:15:172
729299211,10cyclictest27644-21smart_sda09:05:252
729299211,10cyclictest2656-21irqstats11:50:172
728199218,4cyclictest8564-1kworker/u9:111:53:500
728199215,4cyclictest18172-21ls12:25:200
728199214,9cyclictest26309-21grep10:15:260
728199214,9cyclictest26309-21grep10:15:260
728199211,4cyclictest0-21swapper/010:43:180
728199211,11cyclictest0-21swapper/011:27:290
728199211,11cyclictest0-21swapper/011:18:200
728199211,11cyclictest0-21swapper/011:09:090
729899206,3cyclictest0-21swapper/309:40:043
729899201,4cyclictest0-21swapper/311:09:423
729899201,11cyclictest10029-21/usr/sbin/munin10:55:163
729299203,8cyclictest87550irq/128-enp1s0-11:02:232
729299202,9cyclictest23193-21grep12:35:262
729299202,9cyclictest10988-21/usr/sbin/munin08:30:142
729299201,9cyclictest28683-21sshd11:35:222
729299201,6cyclictest27769-21cat07:55:002
729299201,6cyclictest21425-21awk12:35:002
729299201,6cyclictest21425-21awk12:35:002
729299201,6cyclictest20690-21memory08:50:192
729299201,6cyclictest19510-21diskstats07:35:142
729299201,6cyclictest15758-21grep07:25:182
729299201,6cyclictest0-21swapper/212:03:572
729299201,5cyclictest27602-21head10:20:152
729299201,10cyclictest0-21swapper/207:45:242
728199205,8cyclictest24217-21sshd10:10:320
728199205,8cyclictest21971-21/usr/sbin/munin08:55:220
728199205,7cyclictest31327-21cut10:30:020
728199205,7cyclictest27693-21cpuspeed_turbos11:35:120
728199204,9cyclictest14769-21/usr/sbin/munin07:25:180
728199204,8cyclictest3414-21tail09:25:150
728199204,2cyclictest31219-21kworker/0:108:05:480
728199201,7cyclictest12000-21grep11:00:010
728199201,7cyclictest0-21swapper/011:22:130
728199201,7cyclictest0-21swapper/010:51:280
728199201,6cyclictest21692-21df07:40:130
728199201,5cyclictest6264-21sensors_temp09:30:210
728199201,11cyclictest8458-21/usr/sbin/munin12:05:170
728199201,11cyclictest4043-21/usr/sbin/munin11:55:110
728199201,11cyclictest32008-21/usr/sbin/munin11:45:220
728199201,11cyclictest15999-21/usr/sbin/munin09:55:110
728199201,10cyclictest6261-21grep12:00:130
728199201,10cyclictest17772-21irqstats07:30:160
728199201,10cyclictest16943-21sed08:40:260
729899198,3cyclictest0-21swapper/311:52:223
729899197,3cyclictest0-21swapper/310:22:143
729899196,2cyclictest87650irq/129-enp1s0-10:25:223
729899194,3cyclictest0-21swapper/311:00:213
729899191,4cyclictest0-21swapper/310:45:243
729899191,16cyclictest0-21swapper/309:25:223
729899191,10cyclictest0-21swapper/312:35:023
729299195,3cyclictest87350irq/126-enp1s0-10:40:162
729299194,3cyclictest3186-21head08:10:172
729299193,4cyclictest14020-21cstates09:50:132
729299191,9cyclictest87350irq/126-enp1s0-08:05:252
729299191,9cyclictest28348-21diskmemload11:31:512
729299191,9cyclictest28348-21diskmemload09:25:142
729299191,9cyclictest18200-21/usr/sbin/munin10:00:162
729299191,5cyclictest16101-21basename08:40:172
729299191,12cyclictest0-21swapper/207:15:222
729299191,11cyclictest0-21swapper/211:26:552
728199196,8cyclictest0-21swapper/008:50:130
728199196,3cyclictest5626-21sendmail_mailqu08:15:230
728199194,8cyclictest25027-21sort09:00:180
728199194,8cyclictest21638-21/usr/sbin/munin12:35:170
728199192,9cyclictest13523-21hddtemp_smartct12:15:160
728199192,9cyclictest0-21swapper/009:14:090
728199191,9cyclictest19161-21/usr/sbin/munin07:35:220
728199191,6cyclictest12562-21df11:00:130
7281991915,2cyclictest0-21swapper/007:45:260
7281991914,3cyclictest11979-21grep08:30:200
728199191,12cyclictest0-21swapper/012:23:110
728199191,10cyclictest31395-21/usr/sbin/munin10:30:160
728199191,10cyclictest20688-21expr10:05:110
728199191,10cyclictest19440-21sendmail_mailqu10:00:240
728199191,10cyclictest14868-21ntp_states09:50:200
728199191,10cyclictest0-21swapper/009:44:440
728199191,10cyclictest0-21swapper/009:19:570
7281991910,3cyclictest27357-21needreboot09:05:200
729899181,13cyclictest0-21swapper/309:00:403
729299185,3cyclictest5562-21ps08:15:222
729299180,6cyclictest14030-21latency08:35:182
728199185,3cyclictest6184-21grep10:45:150
728199185,3cyclictest27367-21fschecks_count10:20:150
728199184,4cyclictest1506-21taskset09:20:170
728199183,12cyclictest7479-21/usr/sbin/munin07:10:230
728199181,9cyclictest25753-21/usr/sbin/munin07:50:160
728199181,8cyclictest0-21swapper/011:10:260
728199181,4cyclictest8786-21/usr/sbin/munin08:25:250
728199181,4cyclictest1213-21/usr/sbin/munin10:35:160
7281991812,3cyclictest12568-21/usr/sbin/munin07:20:220
728199181,10cyclictest11287-21memory07:15:190
84522170,2sleep30-21swapper/312:05:083
729299173,3cyclictest87550irq/128-enp1s0-08:55:192
728199171,9cyclictest31061-21memory08:00:190
728199171,10cyclictest0-21swapper/008:45:070
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional