You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-17 - 00:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Wed Sep 16, 2026 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1012199270,26cyclictest0-21swapper/011:23:260
10131992623,2cyclictest0-21swapper/309:58:073
1013199260,5cyclictest8143-21munin-run12:35:003
1013199260,1cyclictest0-21swapper/310:53:393
1012199260,25cyclictest4276-21diskmemload09:32:320
10131992522,3cyclictest4276-21diskmemload11:31:483
1013199250,24cyclictest4276-21diskmemload10:45:483
1013199250,24cyclictest21191-21irqstats10:22:313
1013199250,24cyclictest0-21swapper/310:35:133
1013199250,24cyclictest0-21swapper/310:35:133
1013199250,1cyclictest17535-21ssh11:45:533
1013199250,1cyclictest0-21swapper/311:01:463
1013199250,0cyclictest0-21swapper/312:39:353
1013199250,0cyclictest0-21swapper/310:56:443
1013199250,0cyclictest0-21swapper/308:20:023
10128992523,1cyclictest16114-21apt-get10:47:282
10128992520,4cyclictest0-21swapper/210:28:202
10128992520,0cyclictest0-21swapper/211:39:432
10124992523,1cyclictest10049-21apt-get12:37:271
1012499250,24cyclictest12386-21users10:42:371
10121992523,1cyclictest0-21swapper/012:19:440
1012199250,24cyclictest5656-21cron11:35:010
1012199250,24cyclictest21056-21ssh10:51:020
1012199250,24cyclictest0-21swapper/012:35:120
1012199250,24cyclictest0-21swapper/010:29:140
1012199250,24cyclictest0-21swapper/010:04:530
1012199250,1cyclictest22562-21cron09:55:010
1012199250,1cyclictest22562-21cron09:55:010
10131992421,2cyclictest12322-21ssh10:14:253
10131992420,3cyclictest4276-21diskmemload09:42:463
10131992420,3cyclictest0-21swapper/309:07:263
1013199240,23cyclictest4276-21diskmemload10:41:533
1013199240,23cyclictest4276-21diskmemload10:31:053
1013199240,23cyclictest0-21swapper/312:22:403
1013199240,21cyclictest0-21swapper/311:05:563
1013199240,21cyclictest0-21swapper/309:50:203
1013199240,21cyclictest0-21swapper/309:50:203
1013199240,1cyclictest4276-21diskmemload11:59:513
1013199240,1cyclictest4276-21diskmemload10:05:523
1013199240,0cyclictest4276-21diskmemload11:10:363
1013199240,0cyclictest23510-21apt-get09:27:263
10128992423,0cyclictest4276-21diskmemload12:05:132
10128992422,1cyclictest0-21swapper/210:36:372
10128992422,1cyclictest0-21swapper/210:36:372
10128992422,1cyclictest0-21swapper/209:59:502
10128992420,4cyclictest0-21swapper/211:59:392
1012899240,2cyclictest13972-21ssh11:12:572
1012899240,24cyclictest0-21swapper/209:11:452
10124992423,1cyclictest19224-21ssh12:16:301
1012499240,23cyclictest25617-21munin-run10:55:011
1012499240,23cyclictest0-21swapper/111:47:361
1012499240,1cyclictest30025-21kworker/u8:112:13:401
1012499240,1cyclictest0-21swapper/110:23:161
1012499240,0cyclictest0-21swapper/110:16:431
10121992422,1cyclictest1649-21cron11:00:000
1012199242,1cyclictest62550irq/127-eno109:39:370
1012199240,23cyclictest17033-21idleruntime-cro08:30:010
1012199240,23cyclictest12455-21perf12:10:000
1012199240,23cyclictest0-21swapper/010:33:500
1012199240,21cyclictest9146-21apt-get10:12:250
1012199240,0cyclictest4276-21diskmemload09:55:590
10131992323,0cyclictest4276-21diskmemload09:11:383
10131992323,0cyclictest0-21swapper/310:16:203
10131992321,1cyclictest0-21swapper/312:13:093
10131992321,1cyclictest0-21swapper/312:05:573
10131992321,1cyclictest0-21swapper/311:16:373
10131992319,3cyclictest4276-21diskmemload12:16:513
1013199230,23cyclictest0-21swapper/312:00:393
1013199230,23cyclictest0-21swapper/311:42:513
1013199230,23cyclictest0-21swapper/310:02:333
1013199230,22cyclictest8278-21ssh11:37:293
1013199230,22cyclictest27614-21ssh11:25:333
1013199230,22cyclictest1223-21ssh12:28:173
1013199230,0cyclictest0-21swapper/311:52:393
1013199230,0cyclictest0-21swapper/310:28:473
1013199230,0cyclictest0-21swapper/309:47:303
10128992323,0cyclictest0-21swapper/212:34:552
10128992323,0cyclictest0-21swapper/212:00:362
10128992323,0cyclictest0-21swapper/211:29:092
10128992323,0cyclictest0-21swapper/209:27:592
10128992320,2cyclictest4936-21ssh11:05:362
10128992319,3cyclictest1855-21munin-run08:00:002
10128992318,2cyclictest17901-21ssh09:51:042
10128992318,2cyclictest17901-21ssh09:51:042
1012899230,23cyclictest0-21swapper/211:53:182
1012899230,23cyclictest0-21swapper/211:24:262
1012899230,23cyclictest0-21swapper/210:59:342
1012899230,23cyclictest0-21swapper/210:16:242
1012899230,22cyclictest7694-21ssh10:10:152
1012899230,22cyclictest21341-21ssh11:48:482
1012899230,21cyclictest313-21systemd-journal10:50:022
1012899230,0cyclictest19826-21apt-get12:17:242
1012899230,0cyclictest0-21swapper/212:14:412
1012899230,0cyclictest0-21swapper/210:42:152
1012899230,0cyclictest0-21swapper/210:30:562
1012899230,0cyclictest0-21swapper/210:05:532
1012899230,0cyclictest0-21swapper/209:47:562
10124992323,0cyclictest4276-21diskmemload09:26:091
10124992323,0cyclictest0-21swapper/110:00:201
10124992323,0cyclictest0-21swapper/109:18:461
10124992323,0cyclictest0-21swapper/109:10:331
10124992321,1cyclictest0-21swapper/112:31:421
10124992321,1cyclictest0-21swapper/110:28:141
1012499231,21cyclictest0-21swapper/112:26:051
1012499230,23cyclictest0-21swapper/111:37:251
1012499230,23cyclictest0-21swapper/110:59:291
1012499230,22cyclictest5805-21ssh11:35:011
1012499230,22cyclictest4276-21diskmemload12:03:131
1012499230,22cyclictest2631-21latency_hist10:35:001
1012499230,22cyclictest13857-21ssh11:42:311
1012499230,22cyclictest11852-21apt-get08:20:021
1012499230,22cyclictest0-21swapper/111:52:281
1012499230,22cyclictest0-21swapper/111:00:451
1012499230,1cyclictest4276-21diskmemload11:57:271
1012499230,1cyclictest25362-21ssh11:22:431
1012499230,1cyclictest19419-21cron08:35:001
1012499230,1cyclictest151rcu_preempt10:36:251
1012499230,1cyclictest151rcu_preempt10:36:251
1012499230,0cyclictest0-21swapper/112:24:191
1012499230,0cyclictest0-21swapper/109:52:461
1012499230,0cyclictest0-21swapper/109:52:461
1012499230,0cyclictest0-21swapper/109:32:261
10121992323,0cyclictest0-21swapper/012:28:570
10121992322,1cyclictest4276-21diskmemload09:11:030
10121992321,1cyclictest0-21swapper/011:56:000
10121992321,1cyclictest0-21swapper/011:35:330
10121992320,2cyclictest14602-21ssh10:17:200
1012199230,22cyclictest4276-21diskmemload11:07:440
1012199230,22cyclictest4276-21diskmemload11:01:370
1012199230,22cyclictest11271-21ssh09:15:470
1012199230,22cyclictest0-21swapper/010:20:220
1012199230,0cyclictest4276-21diskmemload10:35:360
1012199230,0cyclictest4276-21diskmemload10:35:360
1012199230,0cyclictest4276-21diskmemload10:08:140
1012199230,0cyclictest0-21swapper/011:51:270
1012199230,0cyclictest0-21swapper/011:15:280
1012199230,0cyclictest0-21swapper/010:42:270
1012199230,0cyclictest0-21swapper/009:44:280
1012199230,0cyclictest0-21swapper/009:27:000
10131992222,0cyclictest0-21swapper/309:15:033
10131992222,0cyclictest0-21swapper/307:37:263
10131992221,1cyclictest28846-21ssh09:32:063
1013199220,2cyclictest1842-21awk08:00:003
1013199220,22cyclictest0-21swapper/309:37:073
1013199220,21cyclictest25102-21apt-get07:42:273
1013199220,21cyclictest0-21swapper/308:32:263
1013199220,1cyclictest25829-21ssh11:23:193
1013199220,0cyclictest0-21swapper/309:20:053
10128992222,0cyclictest0-21swapper/212:25:282
10128992222,0cyclictest0-21swapper/211:40:082
10128992222,0cyclictest0-21swapper/210:03:532
10128992222,0cyclictest0-21swapper/209:41:432
10128992222,0cyclictest0-21swapper/209:15:312
10128992221,1cyclictest4276-21diskmemload09:22:122
10128992221,1cyclictest0-21swapper/211:17:222
1012899220,2cyclictest22670-21ssh10:23:232
1012899220,21cyclictest4276-21diskmemload11:00:462
1012899220,21cyclictest0-21swapper/211:31:252
1012899220,1cyclictest5086-21apt-get08:07:272
1012899220,1cyclictest28338-21ssh09:31:192
1012899220,1cyclictest24495-21ssh12:20:352
1012899220,1cyclictest12473-21ssh12:38:192
1012899220,1cyclictest0-21swapper/209:37:282
10124992222,0cyclictest0-21swapper/111:27:121
10124992222,0cyclictest0-21swapper/109:41:051
10124992222,0cyclictest0-21swapper/108:42:281
10124992221,1cyclictest0-21swapper/112:07:591
10124992221,1cyclictest0-21swapper/110:05:401
10124992221,0cyclictest0-21swapper/111:05:271
1012499220,21cyclictest0-21swapper/109:38:391
1012499220,21cyclictest0-21swapper/107:27:251
1012499220,1cyclictest16583-21chrt09:20:241
1012499220,1cyclictest15992-21ssh09:48:371
1012499220,1cyclictest0-21swapper/111:17:221
1012499220,1cyclictest0-21swapper/111:11:591
1012499220,0cyclictest0-21swapper/110:45:541
10121992222,0cyclictest0-21swapper/011:42:340
10121992220,1cyclictest24681-21cron08:45:000
10121992220,1cyclictest0-21swapper/012:20:510
1012199220,22cyclictest4276-21diskmemload09:45:070
1012199220,22cyclictest10381-21ssh11:10:290
1012199220,21cyclictest27053-21turbostat07:45:000
1012199220,21cyclictest0-21swapper/007:22:260
1012199220,21cyclictest0-21swapper/007:22:260
1012199220,1cyclictest0-21swapper/011:25:580
1012199220,1cyclictest0-21swapper/009:22:550
1012199220,0cyclictest0-21swapper/012:00:580
10131992121,0cyclictest0-21swapper/307:47:263
1013199210,2cyclictest0-21swapper/308:52:263
1013199210,21cyclictest0-21swapper/308:07:273
1013199210,1cyclictest22701-21apt-get08:42:273
1013199210,0cyclictest29940-21apt-get08:57:273
1013199210,0cyclictest0-21swapper/308:27:273
10128992120,0cyclictest12702-21apt-get08:22:272
10128992119,1cyclictest4301-21munin-run09:10:012
1012899210,20cyclictest10528-21apt-get07:12:272
10124992120,0cyclictest0-21swapper/108:27:271
10121992121,0cyclictest0-21swapper/011:45:160
10121992119,1cyclictest9879-21apt-get08:17:250
10131992020,0cyclictest0-21swapper/308:02:273
10131992018,1cyclictest0-21swapper/307:15:003
10131992017,3cyclictest0-21swapper/307:22:253
10131992017,3cyclictest0-21swapper/307:22:253
1013199200,19cyclictest0-21swapper/307:17:273
1013199200,0cyclictest0-21swapper/307:27:253
10128992020,0cyclictest0-21swapper/207:52:252
10128992020,0cyclictest0-21swapper/207:47:262
10128992018,1cyclictest17060-21munin-run08:30:012
10124992020,0cyclictest0-21swapper/107:32:271
10124992018,1cyclictest27060-21awk07:45:001
1012499200,0cyclictest0-21swapper/108:57:271
1012499200,0cyclictest0-21swapper/107:37:271
10121992020,0cyclictest27498-21apt-get07:47:260
10121992020,0cyclictest0-21swapper/008:37:240
10121992020,0cyclictest0-21swapper/008:12:260
10121992020,0cyclictest0-21swapper/007:27:250
10121992018,1cyclictest606-21dbus-daemon07:35:010
10131991919,0cyclictest0-21swapper/308:17:263
1013199190,2cyclictest0-21swapper/308:37:253
10128991919,0cyclictest0-21swapper/208:37:242
10128991919,0cyclictest0-21swapper/207:17:282
10128991918,0cyclictest17495-21apt-get08:32:262
10128991917,1cyclictest32354-21apt-get09:02:262
10124991919,0cyclictest2336-21apt-get09:07:261
10124991919,0cyclictest0-21swapper/108:37:241
10124991919,0cyclictest0-21swapper/108:17:251
10124991919,0cyclictest0-21swapper/107:57:231
10124991919,0cyclictest0-21swapper/107:22:251
10124991919,0cyclictest0-21swapper/107:22:251
10124991917,1cyclictest4248-21apt-get08:05:021
10124991917,1cyclictest14871-21dump-pmu-power07:20:001
10121991919,0cyclictest12702-21apt-get08:22:270
10121991919,0cyclictest0-21swapper/008:57:270
10121991919,0cyclictest0-21swapper/008:05:020
10121991919,0cyclictest0-21swapper/008:02:270
10131991818,0cyclictest0-21swapper/308:12:273
10131991815,2cyclictest22092-21perf07:35:013
1013199180,2cyclictest975-21ntp_states09:02:323
1013199180,17cyclictest0-21swapper/307:52:323
10128991818,0cyclictest7481-21apt-get08:12:272
10128991818,0cyclictest17736-21apt-get07:27:252
10121991818,0cyclictest0-21swapper/007:52:250
10121991818,0cyclictest0-21swapper/007:12:270
1013199170,1cyclictest0-21swapper/308:47:313
10128991715,1cyclictest24702-21perf08:45:002
10124991717,0cyclictest0-21swapper/107:12:271
10121991717,0cyclictest0-21swapper/008:32:260
10128991614,1cyclictest22528-21apt-get07:37:262
10124991615,0cyclictest29893-21apt-get07:52:251
1012199160,1cyclictest29551-21latency_hist08:55:000
1012199160,1cyclictest12927-21apt-get07:17:270
1012899156,8cyclictest0-21swapper/208:52:272
10128991513,1cyclictest0-21swapper/208:50:002
1012499150,1cyclictest1737-21runrttasks09:03:271
10121991515,0cyclictest22528-21apt-get07:37:270
10121991515,0cyclictest0-21swapper/009:02:260
10121991515,0cyclictest0-21swapper/007:57:230
10128991414,0cyclictest0-21swapper/208:57:272
1012899140,14cyclictest0-21swapper/208:15:092
1012899140,14cyclictest0-21swapper/208:01:182
1012899140,14cyclictest0-21swapper/207:30:302
1012899140,14cyclictest0-21swapper/207:20:272
1012899140,14cyclictest0-21swapper/207:20:272
1012899140,0cyclictest0-21swapper/207:40:412
1012499140,14cyclictest0-21swapper/108:50:521
1012499140,0cyclictest0-21swapper/108:45:071
1012499140,0cyclictest0-21swapper/108:10:511
1012199140,14cyclictest0-21swapper/009:05:080
1012199140,0cyclictest0-21swapper/008:45:270
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional