-
-
Notifications
You must be signed in to change notification settings - Fork 1k
[backend-only] cdp validation #6828
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -329,3 +329,19 @@ func splitProxyAuth(proxyUrl string) (string, string, string) { | |
|
|
||
| return proxyUrl, "", "" | ||
| } | ||
|
|
||
| func TestRemoteCDP(globalConfig GlobalConfig) error { | ||
| cdpPath := globalConfig.GetScraperCDPPath() | ||
| if cdpPath == "" { | ||
| return fmt.Errorf("CDP path is empty") | ||
| } | ||
| if !isCDPPathHTTP(globalConfig) { | ||
| // unable to test non-http CDP | ||
| return fmt.Errorf("Unable to test non-http CDP paths") | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So, should this be an error? I'm not sure on this one but it seems to me that the path could be valid but just not tested. It would return Perhaps it should be info level or something unless it's actually invalid.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. using the current way to fetch it, it cant handle it. We would need to do an actual xpath scrape to validate PATHs. |
||
| } | ||
| _, err := getRemoteCDPWSAddress(context.Background(), cdpPath) | ||
| if err != nil { | ||
| return fmt.Errorf("Failed to get remote CDP websocket address: %v", err) | ||
| } | ||
| return nil | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be good to accept an input parameter here so that users can test before saving? Currently you cant test and it just saves regardless of it it is correct or not.
Should also do camel case here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
debated doing that as well