You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-26 - 16:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot3s.osadl.org (updated Fri Jun 26, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
107302236165,22sleep10-21swapper/107:05:451
107302236165,22sleep10-21swapper/107:05:451
109822204166,25sleep30-21swapper/307:09:013
109822204166,25sleep30-21swapper/307:09:003
107162203168,23sleep20-21swapper/207:05:342
107162203168,23sleep20-21swapper/207:05:342
108062200163,25sleep00-21swapper/007:06:450
108062200163,25sleep00-21swapper/007:06:450
2340121480,5sleep31115799cyclictest11:15:173
2187721460,3sleep01113699cyclictest12:25:170
11152999490,3cyclictest0-21swapper/212:33:342
11152996259,2cyclictest0-21swapper/209:05:462
10342590,2sleep00-21swapper/007:55:290
42592570,5sleep11114599cyclictest09:20:001
160042460,2sleep10-21swapper/111:00:021
1115799440,30cyclictest0-21swapper/307:11:443
1113699424,24cyclictest1291-21/usr/sbin/munin11:40:210
1115799411,29cyclictest0-21swapper/311:27:523
1115799410,30cyclictest0-21swapper/310:06:163
11157994027,8cyclictest0-21swapper/308:14:353
1115799400,30cyclictest0-21swapper/310:28:233
11157993927,3cyclictest0-21swapper/309:41:523
1113699393,30cyclictest18928-40dnf08:39:050
1113699391,24cyclictest10136-21tail10:45:160
11157993824,9cyclictest0-21swapper/310:34:593
11157993723,8cyclictest0-21swapper/311:56:003
1113699378,19cyclictest0-21swapper/009:30:190
1113699371,28cyclictest87350irq/126-enp1s0-09:40:170
1113699371,27cyclictest10451-21df_inode12:00:140
1113699371,21cyclictest0-21swapper/009:59:230
11152993634,1cyclictest0-21swapper/208:10:192
1113699362,24cyclictest32439-21/usr/sbin/munin09:10:150
1113699361,23cyclictest11068-21/usr/sbin/munin09:35:150
1113699360,26cyclictest0-21swapper/008:00:230
11157993522,7cyclictest0-21swapper/311:31:443
11157993517,3cyclictest0-21swapper/308:50:523
1115799351,26cyclictest0-21swapper/309:29:243
1115799351,26cyclictest0-21swapper/309:29:243
1113699355,20cyclictest827-21runrttasks08:14:230
1113699351,31cyclictest4789-21runrttasks10:34:120
1113699351,28cyclictest22841-21if_err_enp1s010:00:180
1113699351,21cyclictest25756-21unixbench_multi10:05:250
1113699350,25cyclictest0-21swapper/012:35:250
1113699343,22cyclictest3863-21/usr/sbin/munin08:05:240
1113699343,17cyclictest27082-21/usr/sbin/munin11:25:180
1113699341,30cyclictest28034-21/usr/sbin/munin09:00:170
1113699341,25cyclictest17789-21seq11:01:320
1113699341,24cyclictest21291-21/usr/sbin/munin11:10:170
1113699341,22cyclictest87350irq/126-enp1s0-07:40:250
1113699341,22cyclictest0-21swapper/012:30:140
1113699341,21cyclictest17275-21runrttasks09:48:010
1113699341,20cyclictest790-21in:imjournal08:35:010
1113699341,18cyclictest30794-21sed10:20:010
1115799331,24cyclictest0-21swapper/309:12:203
1115799331,20cyclictest0-21swapper/311:48:243
1115799331,20cyclictest0-21swapper/311:48:243
1115799331,19cyclictest0-21swapper/310:24:513
1115799330,20cyclictest0-21swapper/307:53:153
1113699331,24cyclictest799-21taskset11:37:110
11136993312,18cyclictest19057-21grep12:20:130
1113699331,18cyclictest87350irq/126-enp1s0-10:50:110
11136993310,11cyclictest28450-21awk10:15:010
1113699330,21cyclictest6154-21grep11:50:140
11157993221,7cyclictest0-21swapper/309:21:243
11157993219,8cyclictest0-21swapper/312:17:473
11157993218,7cyclictest0-21swapper/309:36:083
1115799321,19cyclictest0-21swapper/312:26:473
1115799320,22cyclictest0-21swapper/312:20:393
1115799320,20cyclictest0-21swapper/311:23:483
1113699329,20cyclictest10449-21/usr/sbin/munin08:20:250
1113699324,14cyclictest5553-21sadc09:20:210
1113699322,26cyclictest18934-21smart_sda09:50:240
1113699321,28cyclictest13887-21/usr/sbin/munin10:55:230
1113699321,19cyclictest827-21runrttasks11:46:540
1113699321,19cyclictest827-21runrttasks11:46:540
11157993118,8cyclictest0-21swapper/307:41:193
1115799311,21cyclictest0-21swapper/312:34:433
1115799311,21cyclictest0-21swapper/309:02:163
1115799311,13cyclictest0-21swapper/310:42:553
1115799310,19cyclictest0-21swapper/308:29:323
1115299311,23cyclictest0-21swapper/210:50:102
1115299311,21cyclictest0-21swapper/207:51:322
1113699316,17cyclictest17964-21mailstats12:15:240
1113699311,27cyclictest8254-21/usr/sbin/munin08:15:200
1113699311,27cyclictest8254-21/usr/sbin/munin08:15:190
1113699311,27cyclictest15484-21chrt07:15:270
1113699311,26cyclictest12609-21smart_sda07:10:220
1113699311,21cyclictest87350irq/126-enp1s0-11:33:450
1113699311,21cyclictest810-21sshd08:41:210
11136993112,16cyclictest17273-21kernelversion07:20:180
1113699311,17cyclictest0-21swapper/009:25:200
1113699311,17cyclictest0-21swapper/009:25:200
1115799301,20cyclictest0-21swapper/311:13:483
1115799301,17cyclictest0-21swapper/310:04:283
1115799301,12cyclictest0-21swapper/312:08:203
1115799301,12cyclictest0-21swapper/308:40:483
1115799300,14cyclictest0-21swapper/311:39:403
1115799300,14cyclictest0-21swapper/309:51:003
1115299301,17cyclictest0-21swapper/210:38:062
1113699309,18cyclictest27466-21cpu07:45:130
1113699307,15cyclictest0-21swapper/007:35:180
1113699302,25cyclictest30881-21smart_sda07:50:230
1113699301,26cyclictest7908-21/usr/sbin/munin11:55:100
1113699301,26cyclictest25585-21sort11:20:160
1113699301,26cyclictest23640-21/usr/sbin/munin08:50:190
1113699301,25cyclictest22093-21smart_sda07:30:260
1113699301,20cyclictest87550irq/128-enp1s0-11:15:260
1113699301,20cyclictest7986-21/usr/sbin/munin10:40:150
1113699300,28cyclictest0-21swapper/008:45:210
211312290,2sleep10-21swapper/109:55:241
1115799299,9cyclictest0-21swapper/309:07:283
11157992912,9cyclictest0-21swapper/310:17:073
1115799291,21cyclictest0-21swapper/308:21:163
1115799291,20cyclictest13451-21munin-node08:02:113
1115799291,13cyclictest0-21swapper/311:07:123
1115799291,13cyclictest0-21swapper/309:34:363
1115799291,10cyclictest32218-21diskmemload09:49:123
1115799290,23cyclictest0-21swapper/309:16:323
1115799290,21cyclictest0-21swapper/310:59:403
1115799290,13cyclictest0-21swapper/310:37:033
1115799290,12cyclictest0-21swapper/310:12:313
1115799290,10cyclictest0-21swapper/307:20:183
1115299291,2cyclictest0-21swapper/209:27:262
1115299291,2cyclictest0-21swapper/209:27:262
1115299291,13cyclictest0-21swapper/211:17:182
1115299291,12cyclictest32218-21diskmemload10:08:312
1115299291,10cyclictest32218-21diskmemload11:14:502
1113699297,19cyclictest5138-21cpu10:35:130
1113699291,25cyclictest2279-21/usr/sbin/munin09:15:210
1113699291,25cyclictest18298-21/usr/sbin/munin11:05:110
1113699291,25cyclictest16197-21sshd12:11:220
1113699291,25cyclictest12320-21munin-node12:05:120
1113699291,21cyclictest1759-21ntp_offset10:25:200
1113699291,17cyclictest25836-21/usr/sbin/munin08:55:150
11136992911,15cyclictest0-21swapper/010:20:160
1113699290,26cyclictest12653-21/usr/sbin/munin08:25:140
11157992814,3cyclictest0-21swapper/307:31:313
1115799281,20cyclictest0-21swapper/307:27:473
1115799281,13cyclictest0-21swapper/312:14:193
1115799281,11cyclictest0-21swapper/312:04:243
1115799280,19cyclictest0-21swapper/308:47:003
1115799280,19cyclictest0-21swapper/307:16:403
1115799280,18cyclictest0-21swapper/311:03:163
1115299281,21cyclictest0-21swapper/211:34:422
1115299281,17cyclictest0-21swapper/209:16:542
1115299281,12cyclictest1644-21df11:40:122
1115299281,12cyclictest0-21swapper/207:57:222
1115299280,13cyclictest0-21swapper/208:58:342
1113699281,24cyclictest18629-21/usr/sbin/munin07:25:130
1113699281,23cyclictest30867-21tr09:05:150
1115799271,18cyclictest0-21swapper/312:36:313
1115799271,18cyclictest0-21swapper/310:52:363
1115799271,11cyclictest0-21swapper/310:49:073
1115799271,10cyclictest0-21swapper/307:56:353
1115799271,10cyclictest0-21swapper/307:45:313
1115299271,9cyclictest0-21swapper/208:49:302
1115299271,12cyclictest0-21swapper/211:01:522
1115299271,12cyclictest0-21swapper/209:42:252
1115299271,11cyclictest0-21swapper/211:27:222
1115299271,10cyclictest0-21swapper/210:11:432
1115299270,11cyclictest0-21swapper/212:27:522
1115299270,11cyclictest0-21swapper/209:20:542
1115299270,10cyclictest0-21swapper/209:57:432
11157992615,6cyclictest0-21swapper/307:35:193
1115799261,11cyclictest0-21swapper/311:44:523
1115799261,10cyclictest0-21swapper/308:18:193
1115799261,10cyclictest0-21swapper/308:18:193
1115799260,12cyclictest0-21swapper/311:54:363
1115799260,12cyclictest0-21swapper/308:36:043
1115299261,9cyclictest0-21swapper/207:46:362
1115299261,11cyclictest0-21swapper/211:55:032
1115299261,11cyclictest0-21swapper/208:39:522
1115299261,11cyclictest0-21swapper/207:34:292
1115299261,10cyclictest0-21swapper/212:20:292
1115299261,10cyclictest0-21swapper/211:48:072
1115299261,10cyclictest0-21swapper/211:48:072
1115299261,10cyclictest0-21swapper/210:45:442
1115299261,10cyclictest0-21swapper/208:40:302
1115299260,9cyclictest0-21swapper/208:18:462
1115299260,9cyclictest0-21swapper/208:18:462
1115299260,11cyclictest0-21swapper/209:33:512
1115799257,9cyclictest0-21swapper/308:05:593
1115799256,3cyclictest20780-21memory09:55:203
1115799251,8cyclictest0-21swapper/308:57:523
1115299251,11cyclictest0-21swapper/212:36:562
1115299251,11cyclictest0-21swapper/209:49:552
1115299251,11cyclictest0-21swapper/208:54:422
1115299250,23cyclictest0-21swapper/209:37:172
1115299250,12cyclictest0-21swapper/212:01:172
1115299250,11cyclictest0-21swapper/211:37:042
1115299250,11cyclictest0-21swapper/210:16:212
1115299250,11cyclictest0-21swapper/209:11:142
1115299250,10cyclictest0-21swapper/210:24:082
1115299241,9cyclictest0-21swapper/208:31:402
1115299241,12cyclictest7962-21hddtemp_smartct10:40:162
1115299241,12cyclictest0-21swapper/209:02:222
1115299241,11cyclictest0-21swapper/207:28:332
1115299240,17cyclictest0-21swapper/210:29:442
1115299240,12cyclictest0-21swapper/212:09:052
11145992418,3cyclictest4187-21hddtemp_smartct11:45:151
11145992418,3cyclictest4187-21hddtemp_smartct11:45:151
11145992414,5cyclictest0-21swapper/108:57:281
1114599241,16cyclictest0-21swapper/110:36:291
1115799239,3cyclictest0-21swapper/308:31:043
1115299231,9cyclictest0-21swapper/208:23:442
1115299231,7cyclictest0-21swapper/207:11:322
1115299230,10cyclictest0-21swapper/207:35:172
1115299221,9cyclictest0-21swapper/207:15:552
1115299221,16cyclictest0-21swapper/209:50:332
1115299221,11cyclictest0-21swapper/207:41:552
1115299221,10cyclictest0-21swapper/208:08:322
1115299220,11cyclictest0-21swapper/212:16:012
1114599222,3cyclictest6375-21sshd09:24:131
11145992215,4cyclictest18928-40dnf08:39:051
1114599221,3cyclictest19645-21munin-run09:55:011
1114599221,2cyclictest827-21runrttasks10:13:191
1115299217,3cyclictest0-21swapper/208:28:002
1115299217,3cyclictest0-21swapper/207:23:492
1115299210,10cyclictest0-21swapper/211:50:372
1114599218,10cyclictest0-21swapper/109:15:001
1114599212,11cyclictest9829-21df_inode10:45:151
1114599211,5cyclictest1689-21strings10:25:191
1114599211,3cyclictest29265-21/usr/sbin/munin09:00:221
11145992112,3cyclictest0-21swapper/107:11:141
1114599211,18cyclictest0-21swapper/112:30:231
1114599211,18cyclictest0-21swapper/110:40:491
1114599211,17cyclictest22497-21cut10:00:161
1114599211,17cyclictest14923-21smartctl07:15:191
1114599211,16cyclictest0-21swapper/109:27:481
1114599211,16cyclictest0-21swapper/109:27:481
1114599211,15cyclictest0-21swapper/110:08:511
11145992111,4cyclictest22747-21kworker/u8:207:39:311
1114599211,10cyclictest0-21swapper/111:58:041
1114599211,10cyclictest0-21swapper/111:12:541
1115299209,3cyclictest0-21swapper/210:01:272
1115299207,3cyclictest0-21swapper/211:20:282
1115299200,10cyclictest0-21swapper/210:59:362
1115299200,10cyclictest0-21swapper/210:32:082
1114599209,3cyclictest16387-21memory09:45:181
1114599207,11cyclictest21811-21kworker/u8:111:00:321
11145992017,2cyclictest0-21swapper/111:20:391
11145992016,3cyclictest0-21swapper/111:38:321
1114599201,2cyclictest28661-21/usr/sbin/munin10:15:181
1114599201,18cyclictest0-21swapper/112:20:141
1114599201,17cyclictest0-21swapper/112:09:111
1114599201,15cyclictest0-21swapper/112:39:591
1114599201,11cyclictest4159-21smart_sda10:30:231
1114599201,11cyclictest32088-21sendmail_mailqu10:20:231
1114599201,11cyclictest14896-21cpu08:30:121
1115299198,4cyclictest0-21swapper/211:05:232
1114599198,3cyclictest19218-21grep07:25:141
11145991915,3cyclictest0-21swapper/111:52:341
11145991915,2cyclictest4925-21ntpq08:05:231
11145991914,3cyclictest21593-21grep08:45:121
1114599191,16cyclictest0-21swapper/107:44:091
1114599191,16cyclictest0-21swapper/107:21:181
1114599191,10cyclictest9211-21sed08:15:191
1114599191,10cyclictest9211-21sed08:15:191
1114599191,10cyclictest3637-21munin-run08:05:011
1114599191,10cyclictest30796-21processes07:50:221
1114599191,10cyclictest23955-21swap11:15:241
1114599191,10cyclictest12512-21kernelversion10:50:181
11145991910,2cyclictest20263-21latency_hist11:10:021
1114599190,18cyclictest0-21swapper/108:10:201
1114599190,18cyclictest0-21swapper/107:58:051
1114599190,17cyclictest0-21swapper/109:40:131
1114599190,17cyclictest0-21swapper/109:06:571
1115299181,11cyclictest0-21swapper/212:12:252
11145991815,2cyclictest0-21swapper/107:47:371
1114599181,4cyclictest24468-21tr08:50:171
1114599181,4cyclictest17339-21head12:15:161
11145991814,3cyclictest0-21swapper/111:30:391
11145991814,3cyclictest0-21swapper/111:25:531
1114599181,16cyclictest0-21swapper/112:15:011
1114599181,16cyclictest0-21swapper/111:40:071
1114599181,16cyclictest0-21swapper/109:36:091
1114599181,16cyclictest0-21swapper/109:31:281
1114599181,16cyclictest0-21swapper/108:40:501
1114599181,16cyclictest0-21swapper/108:29:241
1114599181,15cyclictest0-21swapper/112:00:301
1114599181,11cyclictest0-21swapper/112:27:211
1114599181,10cyclictest10524-21cat08:20:121
11152991712,3cyclictest1655-21/usr/sbin/munin08:00:172
11145991712,3cyclictest21901-21ntp_states07:30:231
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional